Warning: you may do more harm than good if you try to clean a record cover or its label, especially if it’s a matte finish.  If the cover is glossy, one or several of the following cleaning tips may remove unsightly marks.  (Another warning: attempt this at your own risk.) 

1) Remove dirt by wiping with a damp towel. 

 2) Erase markings by rubbing very softly with a rubber eraser. 

 3) Eliminate pen marks by spraying a small amount of hair spray on a cloth, then wiping on cover. 

 4) Remove tape, labels, or stickers by first heating them with a hair dryer. 

 Did we mention: clean the cover only after removing the record from inside? 

Once again, try these only on a glossy cover - even then, you may damage it.  Be careful.