Mount&Blade/Neutral

In addition to the various factions there are also some party types that aren't affected by faction relations.

Bandit and Mercenary troop tree
Bandit parties (shown in red on this tree) are found roaming various parts of the map. They will attack anything they come across. Some bandits do not upgrade at all, while others can eventually upgrade into Mercenaries if you manage to.

Troops from the mercenary line are most easily found in taverns, but they can also be acquired by attacking caravans and then convincing the surviving Caravan Guards to join you. You can also rescue them from parties that have destroyed caravans. Faction parties will regularly attack any caravans that belong to their enemies, so it is quite common to find a faction party carrying a few Caravan Guards.

The end result of the Mercenary line are the heavily-armored Hired Blades. In a fair fight a Hired Blade can take down virtually any other troop type, so they can hold their own unless badly outnumbered or heavily pressed by troops with ranged weapons.

Manhunter troop tree
Manhunters stay close to Looter spawn areas, but they will also attack Sea Raiders and other bandit parties. All troops in the Manhunter line are only equipped with blunt weapons, so if you want to take lots of captives they are perfect for this purpose.

Sword Sister troop tree
Despite slowly evolving from initially vulnerable women, this line results in Sword Sisters, who are extremely powerful fighters on a par with the Hired Blades. Refugees and Peasant Women are very difficult to acquire, so for most players this troop line will be very rare.

If you intend to develop female fighters it is crucial that you keep them out of combat at all costs as they will only be able to hold their own from Camp Huntress onwards (anything lower will be quickly butchered by most other troop types). Once they reach Camp Defender or Sword Sister level they can hold their own against a great many troop types.