I know this question must've been asked a million times, but do I need to read the comics to fully enjoy and appreciate the games?

There's a handful of characters from the comics. Hershel, Shawn (Hershel's son), Glenn, and Jesus. A defected Whisperer is also in the games. There is also the Michonne spin-off that has Michonne, Siddiq, and Elodie (Michonne's daughter). Just know that these characters are NOT the same as the TV version.

I've just started them 

The games (Telltale) are, yes, in the comic universe; however, you don't need to read it to understand. Have fun

Go for it buddy and maybe after read the comics. I never finished all of it I just read what was out when I went through that phase in high school but they were pretty dope.