sanding a layer of oil paint

Not necessarily….it really depends on your technique and how “resilient”​ your sublayers are. A fine grit sandpaper should be fine. I am copying this excerpt from our “Mediums and Additives for Painting” document located in the resources section that might be of interest: 
It is inadvisable to paint a fresh oil painting over an abandoned work in oil as the surface may a) contain a buildup or surface dirt/grime b) contain a layer of protective varnish on the surface and/or c) not provide sufficient “tooth” or absorbency, leading to potential delamination or flaking. In addition, oil films gradually become more transparent as they age which will result in the underlying composition showing through in certain areas. If it is not possible to obtain a new support and start afresh, artists should consider removing any varnish coatings and scraping/sanding down layers of paint (exercising appropriate health and safety precautions) before applying fresh applications of oil paint. It is possible to gently sand the surface of oil/alkyd paint layers. Remember that inhalation of any type of fine, particulate material (particularly if hazardous pigments are present) is not recommended and that a dust mask should be worn during sanding. It is best to avoid and sanding or scraping of grounds containing lead or other toxic pigments. Once you are finished sanding the ground, wipe down the ground using a cloth dampened with odorless mineral spirits to absorb any remaining loose pigments/particles (be sure to properly and properly dispose of the cloth).