We consider the problem of privately learning a sparse model across multiple sensitive datasets, and propose learning individual models locally and privately aggregating them using secure multi-party computation. In this paper, we report some preliminary experiments on distributed sparse linear discriminant analysis, showing both the feasibility and effectiveness of our approach on experiments using heart disease data collected across four hospitals.