*http://trademe.co.nz/* - Pretty much all second hand goods sold through here, also has a lot of "stores" which sell new stuff.  If you're looking at buying anything from AliExpress check here to see if someone's already imported it and you can often just buy from them, skipping the 20 day shipping time
*http://pricespy.co.nz/* - Price aggregator for all tech products, will give you the lowest online price for a specific product or product type. (e.g. cheapest 500gb M.2 SSD)
*http://pbtech.co.nz/* - Pretty good one-stop-shop for anything tech related - pretty good prices overall, I usually buy everything from here 'cause it's easy. Me, James, and Aaron have all bought PC parts / entire PCs from here recently. We have a coupon code ROCKET which may give you a cheaper price
*http://mightyape.co.nz/* - Large online-only store selling lots of different new items, food and drink included. Has a variety of imported goods, esp. from the US. Ask mason to buy it for you and you'll get free overnight delivery with his subscription. Prices vary quite a bit, some things can be very expensive, some things can be very cheap. Check pbtech if they're stocking similar tech items for a lower price
*http://www.briscoes.co.nz/* - One-stop online & physical store for anything home related. Don't buy anything here unless there's a big sale on. Their business model is "Markup everything 500% and do 50% sales every two weeks or so" Is where you can buy name-brand homeware, stuff like sheets, blankets, coffee machines, cutlery etc. There's a big store in dunedin you can buy stuff from. Within walking distance of Dunedin's briscoes is Spotlight, they don't markup as much as Briscoes, but their prices may be more expensive when not on sale, worth a look if you're going into the Dunedin store anyway.
*https://www.tfe.co.nz/* - Specialty and good quality food equipment/utensils. Go here if you want a good knife, or any other kitchen equipment. Fairly good prices, they have an online store, but their actual online store is in Dunedin, close to the Wharf St overbridge, 5min walk from RocketWerkz.
*https://www.bininn.co.nz/bin-inn-dunedin* - James recommended this, is a bulk food/spices/detergent store. Not huge, but good if you want large quantities of things like salt, cooking oil, washing poweder, and possibly hand sanitizer
*https://www.mitre10.co.nz/* - Anything home-improvement related but the dunedin store is massive and they stock pretty much anything that isn't tech or food related. E.g. heaters, light bulbs, plants, paint, mops, cleaning chemicals etc.
And you probably already know about K-Mart and The Warehouse.