This article deals with finite-time adaptive output synchronization of heterogeneous leader-follower multi-agent systems with uncertain dynamics. Two types of uncertainties are considered: 1) uncertainty in the dynamics of follower agents and 2) uncertainty in the leader's trajectory caused by unknown leader dynamics. To this aim, a novel adaptive distributed finite-time observer is first proposed by which the followers can estimate the leader's state and dynamics in finite time. Moreover, the leader's unknown output matrix is estimated in finite time by another distributed dynamic observer. To cope with uncertainty in its own dynamics, then, each follower agent utilizes a novel filtered-regressor-based identifier, decoupled from the leader's state estimator, to learn about its uncertain dynamics without the requirement of measuring the state derivatives. Finally, the presented distributed finite-time observers are leveraged by an adaptive distributed controller to guarantee finite-time output synchronization of each follower agent to the leader. To show the efficiency of the proposed approach, a simulation example is provided.