During the early days of the Federal Convention in 1787, the delegates agreed that their plan for a new government would include a national judiciary. Article III of the Constitution drafted that summer established a Supreme Court and left for the Congress to determine whether other federal courts would be part of the new nation's judiciary. The provision for tenure during good behavior and a prohibition on reductions in salary assured the judges of the Supreme Court and other judges authorized to exercise the judicial power of the United States an independence that the Constitution denied the legislators and president. The constitutional outline of the judiciary, far briefer than the articles defining the legislative and executive branches, offered a general description of the federal courts' jurisdiction. Article III was more specific in its protection of several rights and liberties, such as the guarantee of trial by jury in criminal cases and freedom from bills of attainder or vague charges of treason.

Other articles of the Constitution also shaped the structure and operation of the federal judiciary. According to Article II, the president would appoint judges with the approval of the Senate. In Article I, the enumerated powers of the Congress included the authority "to constitute Tribunals inferior to the Supreme Court." Although Article III made no mention of a chief justice, the provision in Article I for the chief justice to preside in the impeachment trial of a president indicated the delegates' assumption that the Supreme Court would include one leadership position. Article VI required all judges, like state and federal legislators and executives, to be bound by oath to support the Constitution.

The constitutional provisions for the judiciary reflected the conventions' debate on the appointment of judges, the institutional independence of the third branch, and the value of lower federal courts. Many delegates assumed Congress would elect judges, while other wanted the president alone to select the members of the Supreme Court. Madison's original proposal for the Constitution called for the president and members of the Supreme Court to serve on a Council of Revision that would have authority to veto legislation. The most contentious, and finally unresolved, debate concerning the judiciary centered on the proposals for lower federal courts that would operate alongside existing state courts. Supporters of a strong national government wanted a system of lower federal courts with final jurisdiction in many cases, while those who wished to preserve the authority of state governments proposed that the state courts exercise federal jurisdiction on a local level. The debates on the ratification of the Constitution further demonstrated how controversial were the proposals for lower courts and made clear the challenge Congress would face in establishing a national judiciary within a federal system.