In applications of predictive modeling, such as insurance pricing, indirect or proxy discrimination is an issue of major concern. Namely, there exists the possibility that protected policyholder characteristics are implicitly inferred from non-protected ones by predictive models and are thus having an undesirable (and possibly illegal) impact on prices. A technical solution to this problem relies on building a best-estimate model using all policyholder characteristics (including protected ones) and then averaging out the protected characteristics for calculating individual prices. However, such an approach requires full knowledge of policyholders' protected characteristics, which may in itself be problematic. Here, we address this issue by using a multi-task neural network architecture for claim predictions, which can be trained using only partial information on protected characteristics and produces prices that are free from proxy discrimination. We demonstrate the proposed method on both synthetic data and a real-world motor claims dataset, in which proxy discrimination can be observed. In both examples we find that the predictive accuracy of the multi-task network is comparable to a conventional feed-forward neural network, when the protected information is available for at least half of the insurance policies. However, the multi-task network has superior performance in the case when the protected information is known for less than half of the insurance policyholders.