In this paper, we investigate the adaptive PBVS trajectory tracking control problem of robot manipulators with an eye-in-hand camera in the presence of unknown robot kinematics and dynamics. Though PBVS control of robots has been studied for a long time, existing results mainly focused on the regulation/positioning problem. In complex environments, however, simply moving the robot end-effector toward a desired pose is probably not sufficient, and for safety reasons, the traveling paths and trajectories have to be taken into account. Keeping this in mind, in current work we propose an observer-based pose trajectory tracking controller to guarantee that the robot manipulator can track a specified pose trajectory without using Cartesian velocity measurements, where adaptive laws are designed to deal with the uncertainties of the robot kinematic and dynamic parameters. To show asymptotical convergence of the pose tracking errors to zero, closed-loop stability analysis based on Lyapunov theory is given. Furthermore, the effectiveness of the proposed PBVS trajectory tracking controller is illustrated using simulation results based on a 2DoFs planar manipulator.