MadDog, I have all the books from him. To read things like Dagon could be too much weird for a start (it is mainly a description of one of his mad dream).
The Color out of Space is good but not really representative of his work. If I was you, I would start with the Call of Ktulu. You'll have a good basic idea of the rest of his stories.
There are mainly novels in his books except for some. Beyond the Wall of Sleep is one of the full stories that impressed me much, quite nightmarish but you'll have to know a bit the universe of Lovecraft before reading it or you won't profit the story.
August Derleth was is closiest friend and his books are the most fidels to the Lovecraft's universe.
You also have some movies who pathetically tried to copy the universe of Lovecraft. Forgot Re-Animator or From Beyond (both from Stuart Gordon IIRC), they are too far away from the story.
In the Mouth of Madness is maybe the movie who get close from Lovecraft's spirit as well as one or 2 old movies of the 50's or 60's.
Starring Sam Neil.
@Monk: thanks for the link, added to favorite.
@Jafo: IIRC, he died from illness. Must be in the site.
If the critic is good, this game could be one of the few I might consider buying.
Edit: added cover of movie.