Federated learning (FL) can be a good fit when data cannot easily be centralized—for example in healthcare, finance, mobile devices, or across multiple organizations. The core idea is that each participant trains a model locally and only shares model updates (or other aggregated information), not the raw data.
That said, "data never leaves the device" does not automatically mean privacy is guaranteed. There are several practical considerations.
Common frameworks
Some of the most widely used open-source frameworks include:
-
Flower
- Python-first and framework-agnostic (PyTorch, TensorFlow, JAX, etc.)
- Easy to prototype
- Supports simulation and deployment
- Good choice for research and production pilots
-
OpenFL
- Designed for cross-organization collaborations
- Strong adoption in healthcare research
- Focus on secure collaboration between institutions
-
TensorFlow Federated
- Research-oriented
- Deep integration with TensorFlow
- Excellent for experimenting with new FL algorithms
-
FedML
- Supports distributed and federated ML
- Includes benchmarking and deployment tooling
- Good if you're comparing many FL algorithms
Choosing the right federated setting
Different applications call for different architectures.
| Scenario | Typical choice |
|---|
| Mobile phones | Cross-device FL |
| Multiple hospitals | Cross-silo FL |
| Banks collaborating | Cross-silo FL |
| Edge devices in factories | Cross-device or hierarchical FL |
Cross-device FL may involve thousands or millions of intermittent clients, while cross-silo FL usually has a smaller number of stable organizations.
Practical challenges ("gotchas")
1. Non-IID data
This is probably the biggest issue.
Each client's data often has a different distribution:
- One hospital sees mostly elderly patients.
- Another specializes in pediatrics.
- One phone owner takes mostly landscape photos.
Standard averaging (FedAvg) may converge slowly or produce a weaker global model.
Common mitigations include:
- FedProx
- Scaffold
- Personalized federated learning
- Client clustering
2. Communication costs
Training is often cheap compared to communication.
Uploading millions of parameters every training round becomes expensive.
Typical techniques:
- Gradient compression
- Quantization
- Sparse updates
- Fewer communication rounds
3. Client availability
Real devices disconnect frequently.
You need to handle:
- Offline clients
- Slow clients ("stragglers")
- Partial participation
- Interrupted training
Production systems rarely expect every client to participate in every round.
4. Privacy leakage
Model updates can sometimes reveal information about training data through attacks such as gradient inversion or membership inference.
To reduce this risk, consider combining FL with:
- Secure aggregation (so the server sees only aggregated updates)
- Differential privacy
- Trusted execution environments (where appropriate)
These techniques generally improve privacy but may reduce model accuracy or increase computational overhead.
5. Poisoning attacks
A malicious participant can intentionally submit harmful updates to:
- degrade the model,
- introduce backdoors, or
- bias predictions.
Robust aggregation methods (such as median-based or trimmed-mean approaches), anomaly detection, and client reputation systems can help mitigate these risks.
6. Heterogeneous hardware
Clients may have:
- different GPUs,
- CPUs only,
- varying RAM,
- different software versions.
The training process must tolerate diverse compute capabilities.
Evaluation is different
Traditional machine learning assumes you can inspect all data centrally.
Federated learning requires additional metrics:
- Global accuracy
- Per-client accuracy
- Fairness across clients
- Communication rounds
- Bytes transmitted
- Client participation rate
- Training wall-clock time
A model with slightly lower accuracy but much lower communication cost may be preferable in practice.
When federated learning is a good fit
It tends to work well when:
- Regulations prevent sharing raw data.
- Data is naturally distributed.
- Organizations want to collaborate without pooling datasets.
- Privacy is an important design goal.
It may not be the best choice if data can be safely centralized and communication is expensive, or if the participating datasets are extremely small or highly heterogeneous without suitable algorithmic adaptations.
A practical stack
For many new projects, a reasonable starting point is:
- Framework: Flower
- Model: PyTorch model adapted for federated training
- Algorithm: FedAvg initially, then compare with FedProx if client data is highly heterogeneous
- Privacy: Add secure aggregation first; evaluate differential privacy if your privacy requirements call for stronger guarantees
- Experiment tracking: Use tools such as MLflow or Weights & Biases to monitor metrics across federated rounds
For research or proof-of-concept work, this combination offers a relatively gentle learning curve while remaining flexible enough to evolve into more advanced federated learning systems.
