{"data":{"event":{"id":"a1b83019-2586-41d7-8a95-e722e5eecc2b","slug":"flokd-adaptive-knowledge-distillation-for-federated-low-rank-llm-over-wi-d0be7f6058","title":"FLoKD: Adaptive Knowledge Distillation for Federated Low-Rank LLM over Wireless Networks","short_summary":"arXiv:2609.13580v1 Announce Type: new \nAbstract: Large language models (LLMs) have demonstrated strong capabilities across a wide range of natural language processing tasks. However, conventional fine-tuning typically relies on centralized data collection, bringing in privacy concerns. Federated learning (FL) enables collaborative LLM fine-tuning without sharing raw client data, but its deployment over bandwidth-constrained wireless networks is hindered by the communication overhead of model-parameter transmission. Although Low-Rank Adaptation (LoRA) reduces the number of trainable parameters,","full_description":"arXiv:2609.13580v1 Announce Type: new \nAbstract: Large language models (LLMs) have demonstrated strong capabilities across a wide range of natural language processing tasks. However, conventional fine-tuning typically relies on centralized data collection, bringing in privacy concerns. Federated learning (FL) enables collaborative LLM fine-tuning without sharing raw client data, but its deployment over bandwidth-constrained wireless networks is hindered by the communication overhead of model-parameter transmission. Although Low-Rank Adaptation (LoRA) reduces the number of trainable parameters, its communication cost still increases with model scale. Knowledge distillation avoids parameter sharing via output logits, but token-level logits in LLMs incur high communication cost due to sequence length and vocabulary size. Reducing logits lowers the cost but weakens supervision and degrades accuracy. To address these limitations, we propose FLoKD, an adaptive knowledge-distillation framework for federated LoRA fine-tuning of LLMs over wireless networks, which communicates intermediate LoRA activations as the distillation signal rather than logits or full parameters. Since transmitting all blocks over the entire public dataset remains costly, we further propose a transformer block importance scoring framework that selectively transmits the most informative blocks, and two dataset selection strategies that discard public samples deviating from the local data distribution and prioritise those most informative for distillation. Extensive experiments across multiple generative language datasets, including WikiText-103, PTB, and Dialog, demonstrate that our proposed framework reduces communication overhead by 50-65% while achieving rapid convergence to competitive perplexity compared to baselines.\n\nUpdate (2026-09-16): Optimal Pruning for Neural Architectures using Fisher Information Distances. arXiv:2609.16129v1 Announce Type: new \nAbstract: A new scheme for parameter pruning is introduced, derived from the differential-geometric distance in model space. Pruning a parameter sets its value to zero, representing a displacement of the model to the hypersurface on which that parameter vanishes. The minimal distance from the unpruned model to this hypersurface is naturally computed via the geodesic distance in the model space as determined by the Fisher information metric. This distance determines the true change in the model, and its performance, under pruning. By analysing progressively more faithful approximations of this geodesic distance a natural hierarchy of optimality for pruning methods is determined. This starts with the traditional magnitude pruning, then develops into new more sophisticated and effective pruning schemes. The method is demonstrated for both fully-connected networks and vision transformers, on MNIST and CIFAR-10, over the complete $0$-$100\\%$ pruning range and across five random seeds. It outperforms pruning by parameter magnitude and by the local Fis\n\nUpdate (2026-09-17): Beyond Data Filtering: Knowledge Localization for Capability Removal in LLMs We localize dangerous knowledge to a subset of model's parameters, so it can be easily removed after training.. Beyond Data Filtering: Knowledge Localization for Capability Removal in LLMs We localize dangerous knowledge to a subset of model's parameters, so it can be easily removed after training.\n\nUpdate (2026-09-18): Cache-to-Cache: Direct Semantic Communication Between Large Language Models. Article URL: https://arxiv.org/abs/2510.03215\nComments URL: https://news.ycombinator.com/item?id=49758615\nPoints: 12\n# Comments: 0","ledger_type":"benefit","primary_domain_id":"8f1af1b9-7302-4b51-aae9-fdfac04d158a","event_status":"provisional","event_date":"2026-09-16T00:00:00.000Z","discovery_date":"2026-09-16T00:00:00.000Z","first_published_date":"2026-09-16T00:00:00.000Z","last_reviewed_date":"2026-09-18T00:00:00.000Z","geographic_scope":"International","affected_population":null,"base_impact_tier":1,"base_score":"1.00","attribution_multiplier":"0.1000","evidence_multiplier":"0.4000","realization_multiplier":"0.2000","durability_multiplier":"0.5000","current_event_score":"0.004000","confidence_level":"low","score_explanation":"Ingest update: independent source raised evidence; added a follow-up source. Score remains provisional until review. Follow-up ingest: independent source raised evidence; added a follow-up source. Follow-up ingest: independent source raised evidence; added a follow-up source.","methodology_version_id":"8745541b-0f8e-4d5a-9b47-5291d3813a8d","original_methodology_version_id":"8745541b-0f8e-4d5a-9b47-5291d3813a8d","published_at":"2026-09-17T01:00:48.950Z","created_at":"2026-09-17T01:00:48.950Z","updated_at":"2026-09-18T20:01:06.013Z","flags":[],"domain_name":"Biology","domain_slug":"biology","methodology_version":"0.2"},"contributions":[{"id":"5414e499-2ab7-4321-a126-74cbba72aef5","event_id":"a1b83019-2586-41d7-8a95-e722e5eecc2b","model_id":"535014f7-c941-4d39-aa9b-8f9c1d3d07b7","role_description":"Unspecified system mentioned or implied by a news item. Remap to a named release when identified.","attribution_multiplier":"0.1000","credit_share":"1.0000","contribution_score":"0.004000","attribution_rationale":"News ingest does not infer a named model from the publisher alone. Attribution stays unspecified until a release is identified.","attribution_confidence":"medium","first_used_date":"2026-09-16T00:00:00.000Z","model_version_if_known":null,"review_status":"approved","created_at":"2026-09-17T01:00:48.959Z","model_slug":"unspecified-ai-system","model_name":"Unspecified AI system","identity_class":"unknown","is_internal":false,"family_name":"Unspecified","family_slug":"unknown-unspecified","organization_name":"Unknown","organization_slug":"unknown"}],"sources":[{"id":"f59f98ee-f726-4e35-bb43-5f2b1cd00c7f","event_id":"a1b83019-2586-41d7-8a95-e722e5eecc2b","url":"https://arxiv.org/abs/2609.13580","canonical_url":"https://arxiv.org/abs/2609.13580","source_type":"preprint","publisher":"arXiv cs.AI","author":null,"publication_date":"2026-09-16T00:00:00.000Z","retrieved_at":"2026-09-17T01:00:48.969Z","title":"FLoKD: Adaptive Knowledge Distillation for Federated Low-Rank LLM over Wireless Networks","excerpt":"arXiv:2609.13580v1 Announce Type: new \nAbstract: Large language models (LLMs) have demonstrated strong capabilities across a wide range of natural language processing tasks. However, conventional fine-tuning typically relies on centralized data collection, bringing in privacy concerns. Federated learning (FL) enables collaborative LLM fine-tuning without sharing raw client data, but its deployment over bandwidth-constrained wireless networks is hindered by the communication overhead of model-par","content_hash":null,"source_reliability_class":"medium","is_primary_source":true,"is_independent":true,"is_peer_reviewed":false,"archived_url":null,"created_at":"2026-09-17T01:00:48.969Z"},{"id":"d4ce6d30-9c37-47a1-bf9c-480d855efda3","event_id":"a1b83019-2586-41d7-8a95-e722e5eecc2b","url":"https://arxiv.org/abs/2609.16129","canonical_url":"https://arxiv.org/abs/2609.16129","source_type":"preprint","publisher":"arXiv cs.AI","author":null,"publication_date":"2026-09-16T00:00:00.000Z","retrieved_at":"2026-09-17T03:00:57.785Z","title":"Optimal Pruning for Neural Architectures using Fisher Information Distances","excerpt":"arXiv:2609.16129v1 Announce Type: new \nAbstract: A new scheme for parameter pruning is introduced, derived from the differential-geometric distance in model space. Pruning a parameter sets its value to zero, representing a displacement of the model to the hypersurface on which that parameter vanishes. The minimal distance from the unpruned model to this hypersurface is naturally computed via the geodesic distance in the model space as determined by the Fisher information metric. This distance de","content_hash":null,"source_reliability_class":"medium","is_primary_source":false,"is_independent":false,"is_peer_reviewed":false,"archived_url":null,"created_at":"2026-09-17T03:00:57.785Z"},{"id":"67030222-8e22-41f6-be42-f8eec6777ac3","event_id":"a1b83019-2586-41d7-8a95-e722e5eecc2b","url":"https://alignment.anthropic.com/2025/selective-gradient-masking","canonical_url":"https://alignment.anthropic.com/2025/selective-gradient-masking","source_type":"developer_announcement","publisher":"Anthropic Alignment Science","author":null,"publication_date":"2026-09-17T00:00:00.000Z","retrieved_at":"2026-09-17T03:01:11.502Z","title":"Beyond Data Filtering: Knowledge Localization for Capability Removal in LLMs We localize dangerous knowledge to a subset of model's parameters, so it can be easily removed after training.","excerpt":"Beyond Data Filtering: Knowledge Localization for Capability Removal in LLMs We localize dangerous knowledge to a subset of model's parameters, so it can be easily removed after training.","content_hash":null,"source_reliability_class":"medium","is_primary_source":false,"is_independent":true,"is_peer_reviewed":false,"archived_url":null,"created_at":"2026-09-17T03:01:11.502Z"},{"id":"fba3af35-3d1a-447b-9025-87dc34d858c8","event_id":"a1b83019-2586-41d7-8a95-e722e5eecc2b","url":"https://arxiv.org/abs/2510.03215","canonical_url":"https://arxiv.org/abs/2510.03215","source_type":"journalism","publisher":"Hacker News","author":null,"publication_date":"2026-09-18T00:00:00.000Z","retrieved_at":"2026-09-18T20:01:05.986Z","title":"Cache-to-Cache: Direct Semantic Communication Between Large Language Models","excerpt":"Article URL: https://arxiv.org/abs/2510.03215\nComments URL: https://news.ycombinator.com/item?id=49758615\nPoints: 12\n# Comments: 0","content_hash":null,"source_reliability_class":"medium","is_primary_source":false,"is_independent":true,"is_peer_reviewed":false,"archived_url":null,"created_at":"2026-09-18T20:01:05.986Z"}],"claims":[{"id":"6aaf1323-c144-4a9c-880e-0d87118070ad","event_id":"a1b83019-2586-41d7-8a95-e722e5eecc2b","claim_text":"FLoKD: Adaptive Knowledge Distillation for Federated Low-Rank LLM over Wireless Networks","claim_type":"outcome","claim_status":"supported","confidence_score":null,"created_at":"2026-09-17T01:00:48.978Z","updated_at":"2026-09-17T01:00:48.978Z"}],"revisions":[{"id":"4b2f23aa-4b9e-45e5-a552-9e39b0192f89","event_id":"a1b83019-2586-41d7-8a95-e722e5eecc2b","model_id":null,"previous_score":"0.000000","new_score":"0.001000","previous_factors":{},"new_factors":{"evidence":0.1,"base_score":1,"durability":0.5,"attribution":0.1,"realization":0.2},"change_reason":"Auto-published from news ingest.","trigger_type":"news_ingest","trigger_source_ids":null,"reviewer_id":null,"review_status":"published","created_at":"2026-09-17T01:00:48.985Z"},{"id":"059c382d-2e19-4373-a75e-44a7941324a9","event_id":"a1b83019-2586-41d7-8a95-e722e5eecc2b","model_id":null,"previous_score":"0.001000","new_score":"0.002000","previous_factors":{"tier":1,"evidence":0.1,"base_score":1,"durability":0.5,"attribution":0.1,"realization":0.2},"new_factors":{"tier":1,"models":["535014f7-c941-4d39-aa9b-8f9c1d3d07b7"],"evidence":0.2,"base_score":1,"durability":0.5,"attribution":0.1,"realization":0.2},"change_reason":"Ingest update: independent source raised evidence; added a follow-up source.","trigger_type":"ingest_update","trigger_source_ids":["d4ce6d30-9c37-47a1-bf9c-480d855efda3"],"reviewer_id":null,"review_status":"published","created_at":"2026-09-17T03:00:57.804Z"},{"id":"500fe7d9-d2f1-465e-8150-58b0303ade01","event_id":"a1b83019-2586-41d7-8a95-e722e5eecc2b","model_id":null,"previous_score":"0.002000","new_score":"0.003000","previous_factors":{"tier":1,"evidence":0.2,"base_score":1,"durability":0.5,"attribution":0.1,"realization":0.2},"new_factors":{"tier":1,"models":["535014f7-c941-4d39-aa9b-8f9c1d3d07b7"],"evidence":0.3,"base_score":1,"durability":0.5,"attribution":0.1,"realization":0.2},"change_reason":"Ingest update: independent source raised evidence; added a follow-up source.","trigger_type":"ingest_update","trigger_source_ids":["67030222-8e22-41f6-be42-f8eec6777ac3"],"reviewer_id":null,"review_status":"published","created_at":"2026-09-17T03:01:11.523Z"},{"id":"6ae8ba29-f68e-4ea8-af16-07af62ce5e41","event_id":"a1b83019-2586-41d7-8a95-e722e5eecc2b","model_id":null,"previous_score":"0.003000","new_score":"0.004000","previous_factors":{"tier":1,"evidence":0.3,"base_score":1,"durability":0.5,"attribution":0.1,"realization":0.2},"new_factors":{"tier":1,"models":["535014f7-c941-4d39-aa9b-8f9c1d3d07b7"],"evidence":0.4,"base_score":1,"durability":0.5,"attribution":0.1,"realization":0.2},"change_reason":"Ingest update: independent source raised evidence; added a follow-up source.","trigger_type":"ingest_update","trigger_source_ids":["fba3af35-3d1a-447b-9025-87dc34d858c8"],"reviewer_id":null,"review_status":"published","created_at":"2026-09-18T20:01:06.027Z"}],"secondary":[]},"methodology_version":"0.1"}