Do my kittens really need kitten milk, or do they get more than enough of their nutrients they need from their mother? ?

Their mother's milk is best, for thousands of generations mothers milk has been raising healthy kittens, Now if the mother is not healthy and has had a poor diet, then the kittens may need some help. 

Talk to a veterinarian.

If they are getting mother's milk, they need water from you more than they need milk.  Get them used to drinking water.