Potential energy is related to only conservative forces.
A conservative force is a force with the property that the work done in moving a particle between two points is independent of the taken path.
The concept on a conservative force applies to potential energy. Potential energy is associated with forces that act on a body in a way that depends only on the body's position in space.
Forces derivable from a potential are also called conservative forces.