What were Madison's and Jefferson’s main arguments against the Alien and Sedition Acts?

It is not surprising that both Madison and Jefferson were such staunch opponents of the Alien and Sedition Acts. In addition to both being Democratic-Republicans—and therefore naturally hostile to the Adams administration—they had been closely involved in devising the Bill of Rights. Jefferson provided the inspiration, whereas Madison was involved in the drafting.
Both men recognized that the Bill of Rights was inadequate to deal with this raft of authoritarian legislation. They could not get the Supreme Court to rule these laws as unconstitutional because the Court had not yet taken up the power of judicial review. It would not do so until the landmark case of Marbury v Madison five years later.
Thus, they hit upon the idea of the people themselves challenging unauthorized government action  through their state legislatures  The main arguments they used to support this move were a mixture of the practical and the ideal.
Practically speaking, this appeared the only way to challenge the Adams administration. With a Federalist-controlled Congress, it was impossible to repeal the hated legislation. Bringing the matter before state legislatures could allow them to get around this problem.
There was also principle involved. The Alien and Sedition Acts went against the spirit of republican liberty enshrined in the Declaration of Independence. Americans had shed blood on the battlefield fighting against a centralizing, tyrannical power. However, now it appeared that a new tyranny was rising, one built by Americans on American soil.
Jefferson and Madison's arguments were set out, respectively, in the Kentucky and Virginia Resolutions. The federal government had effectively broken its compact with the American people by violating the sacred trust placed in their hands to protect their fundamental rights and liberties. As the people had been party to the original compact which established the federal government in the first place, then it was only right and proper that they should decide when the terms of that agreement had been violated.
Jefferson's language was a good deal more strident than Madison's rhetoric. He openly declared that state legislatures had the right to declare acts of the federal government as null and void. As the Alien and Sedition Acts were unauthorized in his eyes, they effectively had no legal force.
Madison's response in the Virginia Resolution was more measured. Yes, the federal government had openly violated liberty, but Virginia would still retain its professed loyalty to the United States. Madison, in keeping with the principles he had already set out in the The Federalist, looked upon state government not as the ultimate repository of political sovereignty (as Jefferson believed), but as an important instrument for the mobilization of public opinion whenever the federal government exceeded its constitutional bounds.
Madison also spoke eloquently against restrictions placed by the Acts upon freedom of the press. For Madison, the government saw things the wrong way; it was not the government that censored the people, it was the people, exercising their right to free speech, who censored the government. Only in this way could the government ever be held responsible to the American people. This principle was a cornerstone of republican liberty. A free press was also a means by which the public opinion marshaled by the states could be given full expression, bringing the maximum amount of pressure to bear upon the federal government.
https://www.lawliberty.org/liberty-forum/sound-the-alarm-to-the-people-james-madison-thomas-jefferson-and-the-principles-of-1798/