A video showing that everything one needs to play D&D is (likely) available at your local Dollr General (even dice! Though, they seemed to be sold out in the video).
https://youtu.be/HdRrvn6G4Bs
Also, one doesn't necessarily need a battlemap (based on which edition one is playing), but I never noticed that gift wrap paper has a 1 inch square grid on the back.
I buy lots of terrrain stuff at Dollerama. They've got these lovely 1/8 x 8 x 8 boards and cheap paint and plastic plants, and rocks and sand. They used to have these green plastic spiney plants that were awesome but I can't get them anymore. :(
I also got 3 sets of dice for $4 there once but they sold out fast and I haven't seen them restocked since. You can use some of the dinosaurs and such for monsters but they're not so great for characters.
Walmart carries sets of dice. There are six sets in the package, 7 dice in a set, and each set has its own faux cloth bag, all for 8 bucks.
Neat. And I've seen various gaming-sized miniatures in the diorama section of a hobby store. There's an online store called The Game Crafter that sells pieces like glass beads, meeples, toy coins, and customized boards and cards. Also have seen videos about making battle mats and terrain by spraypainting bits of foam packaging and even foam dinner plates.