The first President settled upon the area at the very south of the area designated by Congress (in fact the Residency Act had to be amended slightly to allow the incorporation of Alexandria). By August 1790 Jefferson was drafting tentative plans; Madison and Jefferson visited Georgetown in September and in October Washington himself looked at various sites. During that winter Commissioners were appointed and Andrew Ellicott was asked to begin a detailed survey of the Federal District and to mark its boundaries.
Ellicott began his survey at Jones Point, to the south east of Alexandria, which was a well-known landmark used by navigators, and the square was measured from there. A boundary stone was laid there on April 15th 1791 - and 40 stones laid to mark off the miles on the perimeter of the Federal District. Apparently 38 still remain! Further information can be found at http://www.boundarystones.org/
The District included the Potomac; Eastern Branch (Anacostia River); Tiber Creek (Goose Creek); and Rock Creek, as well as a host of smaller streams. A ridge to the north of Tiber Creek became the site for the President's House (The White House). The western part of Jenkins Hill who chosen for the Capitol Building.
Pierre-Charles L'Enfant, a French engineer and architect, who had come to America in 1777 with Lafayette, submitted plans for the city in June 1791 (having arrived in the area only in March). A number of attempts were made to sell land within the District but these were less successful than had been hoped for. The Cornerstone of the Capitol was laid on 18th September 1793, on a day which began with a parade and concluded with the third public auction of lots (which fared even worse than the previous two).
Washington DC became the Federal Capital on 1st December 1800, but the new city was not yet ready. The Senate wing of the Capitol Building had been built, but only the foundations of the House of Representatives existed. The House had to use the west side of the second floor (the library) of the Senate wing.