Introduction:
Keeping your burden low helps you both stay alive longer and dish out more damage. And its (mostly) free!
While most (if not all) of these informations can be found elsewhere, i'll do my best to sum it up briefly.

This is written with young characters in mind, but some items mentioned might require a bit of saving up. To keep things simple it will assume dodge/dagger users, but it translates well to other weapon and defense classes. We wont cover armour here, so if you have 18+ strength and use parry, this might not be that important for you.

Obvious things first:
Money is heavy. Its a great idea to bank often or not loot pence at all.
Loot is heavy. Fence often, and loot smartly. In general, jewellery and silk clothes are worth taking. Its a good idea to know the fence locations around your hunting grounds.

Held things burden you by 1.5 their weight, so dont hold things when you are climbing, swimming, balancing, and similar.
Worn things burden you only by half their weight. So put everything you dont need immedeately in your bags and pockets!

Reduction:
What do you really need to carry around?
Weapons, clothes to keep you warm, a light source, some healing and a container. Sheathes can be very light and are quite convenient too.
Even lightweight stuff adds up quickly. Try vaulting everything besides what you really need and see your burden go down.

Optimisation:
Weapons: Using a rather light weapon is a great idea. Try using stilettos, tiger fangs, collapsible steel battons and similar. Remember, a weapon held burdens you three times as much as a worn thing!

Clothes: Going naked is not for everyone and every season. A cloak or coat should serve you well. A silk or spidersilk cloak from Nicolette is both stylish and light, while also providing a bit of storage space and a bit or warmth. For winter (or desert excursions) there are highly efficient cloaks sold at the Magical Emporium (red for winter, white for desert).

Sheathes: Black boots (from assassins) can hold 2 daggers, Sword belts (moonlit market) hold 2 swords or daggers, belt loops (from thieves) for 1 mace each.
Light source: depending on budget, stone rings or permalights are good.
Healing: bandages are great, but until you get there, a wearable canteen is a lot lighter than a bottle.
Containers: Black velvet bags are best lbs/lb, SWAG bags are good too. (thieves guild both)