Concrete ML v1.7: Transformer Model Fine-tuning and DNA Ancestry Applications

Concrete ML v1.7 introduces several interesting new features. Users can now fine-tune LLMs and neural networks on encrypted data, unlocking more potential privacy-preserving machine learning use cases. Concrete v1.7 also adds GPU acceleration that enables faster performance - up to 1-2x speed-up for large neural networks on server-grade GPUs such as the NVIDIA H100. A new Hugging Face space showcases a complex use-case: predicting ancestry from encrypted DNA. Finally, Concrete ML now supports Python 3.11 and PyTorch 2.

Fine-tuning LLMs on encrypted data

Fine-tuning large language models is a powerful technique to tailor models to specific domains and tasks. Concrete ML v1.7 makes it possible to fine-tune LLMs and neural networks on encrypted data. Using low rank approximation parameter-efficient fine-tuning, this feature allows models to adapt while maintaining data privacy. While a small set of private fine-tuned parameters are kept on the user-side, the computations involving large weight matrices are securely outsourced to remote servers. This feature is demonstrated in a fine-tuning GPT2 demo, where the model learns about Fully Homomorphic Encryption (FHE) - a topic not in its original knowledge base.

GPU acceleration for encrypted inference

Concrete ML v1.7 boosts performance for encrypted inference with GPU acceleration, particularly when working with large models like GPT, ResNet, or VGG. By leveraging large GPUs such as NVIDIA H100 or A100, users can obtain notable speed-ups. For instance, encrypted inference using the ResNet18 model on the H100 GPU is 1.2x faster than on a 192-core CPU. For a VGG model on CIFAR, it brings up to 2x faster performance. When compared to a desktop CPU, the H100 GPU achieves a 5-10x speed-up on both of these models. 

To install the GPU enabled backed of Concrete ML, simply run the following commands:

pip uninstall concrete-python
pip install --extra-index-url https://pypi.zama.ai/gpu concrete-python

Encrypted DNA Ancestry Demo

DNA testing platforms often analyze users’ genetic data in the clear, leaving sensitive data at risk of exposure. With Concrete v1.7, a new Hugging Face space shows how to perform this analysis on encrypted data. The demo focuses on the 22nd chromosome, which is particularly relevant for tracing human evolutionary history and migration patterns across the world. By analyzing specific markers known as Single Nucleotide Polymorphisms (SNPs) at key positions on this chromosome, this demo infers ancestry using encrypted DNA, while ensuring that all the genetic data remains safe and private.

Additional links

• Star Zama's Concrete ML GitHub repository to endorse our work and help us to reach the 1,000 stars ⭐️ milestone!
• Review the Concrete ML documentation.
• Get support on our community channels.
• Participate in the Zama Bounty Program to get rewards in cash!