"We propose new parametrizations for neural networks in order to estimate extreme quantiles for both non-conditional and conditional heavy-tailed distributions. All proposed neural network estimators feature a bias correction based on an extension of the usual second-order condition to an arbitrary order. We establish convergence rates in terms of the neural network complexity. The finite sample performances of the non-conditional neural network estimator are compared to other bias-reduced extreme-value competitors on simulated data: our method outperforms them in difficult heavy-tailed situations where other estimators almost all fail. Finally, the conditional neural network estimators are implemented to investigate the behavior of extreme rainfalls as functions of their geographical location in the southern part of France. "