It is realized that force exerted at the two ends of an inerter is proportional to their relative acceleration between them. Generally, a mechanical inerter has much smaller mass than a conventional mass damper when identical inertial force is attained due to its mechanical characteristics. In this study, a novel active control system named active inerter damper (AID) is proposed to mitigate seismic response of buildings which is composed of a mechanical ball-screw inerter, a servo-hydraulic actuator, sensors, and a digital controller. Compared to an active mass damper, the AID requires less mass support structure and installation space in real engineering practice. In this paper, two force tracking controllers for the AID system are proposed, designed, and validated in the laboratory first which aim to impose control force on buildings accurately because force tracking performance of an active control system is extremely crucial to its seismic control performance. Afterwards, shake table tests of a three-story steel specimen with an AID system are conducted to validate the feasibility of AID control application for buildings. The control force is computed from a machine learning-based controller that utilizes acceleration feedback directly from the steel specimen. Experimental results demonstrate that force tracking performance of the AID system is satisfactory. Meanwhile, the AID system mitigates seismic response of the steel specimen successfully and effectively.