This study presents a near-optimal learning-based approach for sliding-mode control of uncertain nonsquare nonlinear systems subject to output constraints. To achieve a compromise between safety and optimality, a reinforcement learning algorithm is proposed to compute the near-optimal values of the sliding manifold coefficients. In the reinforcement learning algorithm, only measured input-output data obtained by experiments or simulations are employed. Furthermore, the presented method does not require partial knowledge of the system dynamics to initialize the reinforcement learning process. By employing the tuned sliding vector, an adaptive fuzzy sliding-mode control (AFSMC) input, including a fuzzy term and a robust term, is generated. The fuzzy term is used to approximate an unknown nonlinear function and the robust term is designed for mismatch compensation. To guarantee that the output constraints are not violated, the adaptation laws for obtaining the fuzzy singletons and the bounds of the approximation errors are designed based on the barrier Lyapunov functions (BLF) theorem. The closed loop asymptotic stability is theoretically analyzed in the article, while the effectiveness of the method is assessed in simulation.