Andrew Jackson was a man of the people.  Unlike the bureaucratic Aristocats of the previous presidencies, Jackson wanted what was best for the people.  Although many would criticize many of his actions regarding the nullification crisis, the Indian Removal Act, and the second national bank, I'd argue that they were all were necessary for the development of the government at the time.
For example, people argue the Indian Removal Act to be one of the most egregious actions made by  Jackson during his Presidency.  It gave the President the ability to "exchange" west Indian lands for land in teh east.   Yes, Jackson did trade 100 million acres for 30 million acres, but the transaction did not come without resistance, which would change America for the better.  The Seminole Indians would retaliate with war and win.  Both sides lost hundreds of people, and the US ended with an economic loss of $20 million, but that was it.  The Seminole were allowed to stay, which shows Jackson knew when to quit.  Such knowledge is important, as a president who drives the country into the ground out of pride is bound to cause issues.
In the case of the Cherokee, they went to the Supreme Court to ask to stay on their land and won.  However, Jackson would enforce his executive power to make them leave anyway.  It is easy to see how such actions could appear irredeemable, but consider that, up until this point, the president had never exercised the executive power to its fullest potential.  The immorality of his actions is outweighed by the demonstration of the president's control, which created vital discussions regarding the balance of executive power.
Moving on to the less controversial issue, the Nullification Crisis would cause Jackson a bit of trouble, since it was started by his Vice President.  South Carolina declared that a federal tariff was nullified in their state, so Jackson proposed the Olive Branch and the Sword Agreement, which lowered the tax over time as long as the President could send troops in if the state did this again. The South took the agreement, which in itself is evidence of the decision not being terrible.  Sure, imposing pseudo-martial law in response to a state claiming to nullify a law sounds preposterous but is 100% within the bounds of the Constitution.  If states ignore a federal law that the President is enforcing, then executive power gives him the right to do whatever necessary to enforce it.  To call Jackson's actions here controversial is misguided.  The Olive Branch and the Sword was Jackson's way to restore peace to the worry-ridenn South.
Finally, with the Second National Bank, the question is no longer one of morality but necessity.  Jackson would veto the Second National Bank because he believed it was a corruptable force designed to make the rich richer.  However, some argue that the fall of the Second National Bank would cause the economic panic that marked the beginning of Martin Van Buren's presidency.  Due to a lack of historical evidence and overall acceptance among the historical communities, claiming Jackson's disapproval of the Second National Bank as harmful is entirely wrong.  To do so is like convicting without evidence.  100% legal but amoral and ignorant.
Therefore, Jackson, while certainly not the best president in the world, was justified in what he did.  Had he not responded to each controversy in the same way he did, there could have been disastrous consequences.  Discussion regarding the philosophical implications of executive power may have never occurred.  Jackson could have turned out to be a war driven maniac bent on plunging the US to the ground out of pride.  Worst of all, he could have ignored the dangerous nature of the Second National Bank and given way to the US's horrific downfall, as it fell from a democratic republic into the greedy hands of plutocracy.  Truth can appear to come in many flavors, but the only way to know what it really tastes like is to try them all.