← Papers

Multimodal capabilities in open LLM families arrive as rare founder events and then spread quickly within lineages, but rarely convert text-only models into vision-language descendants; most VLM releases appear as new roots rather than as children of text-generation models.

Founder effects shape the evolutionary dynamics of multimodality in open LLM families

Manuel Cebrian · January 27, 2026

arxiv descriptive medium evidence 7/10 relevance Full text usable extracted full text Source PDF

Structured author observations

Linked only from stored provider relations; the raw author line above is never matched by name.

Arxiv

Latest observation: July 29, 2026

Manuel Cebrian unresolved corpus identity

Semantic Scholar

Latest observation: August 25, 2026

Manuel Cebrian provider ID

Using >1.8M Hugging Face model records, the paper finds multimodality (primarily vision-language) enters open LLM families via rare founder events and then expands rapidly within descendant lineages, while cross-type transfers from text-generation parents to VLM children are extremely uncommon.

Citation observations

Cumulative provider counts captured on specific dates; providers are never combined.

0 cumulative citations

Semantic Scholar · Observed August 25, 2026

View corpus context

Large language model (LLM) families are improving rapidly, yet it remains unclear how quickly multimodal capabilities emerge and propagate within open families. Using the ModelBiome AI Ecosystem dataset of Hugging Face model metadata and recorded lineage fields (>1.8x10^6 model entries), we quantify multimodality over time and along recorded parent-to-child relations. Cross-modal tasks are widespread in the broader ecosystem well before they become common within major open LLM families: within these families, multimodality remains rare through 2023 and most of 2024, then increases sharply in 2024-2025 and is dominated by image-text vision-language tasks. Across major families, the first vision-language model (VLM) variants typically appear months after the first text-generation releases, with lags ranging from ~1 month (Gemma) to more than a year for several families and ~26 months for GLM. Lineage-conditioned transition rates show weak cross-type transfer: among fine-tuning edges from text-generation parents, only 0.218% yield VLM descendants. Instead, multimodality expands primarily within existing VLM lineages: 94.5% of VLM-child fine-tuning edges originate from VLM parents, versus 4.7% from text-generation parents. At the model level, most VLM releases appear as new roots without recorded parents (~60%), while the remainder are predominantly VLM-derived; founder concentration analyses indicate rapid within-lineage amplification followed by diversification. Together, these results show that multimodality enters open LLM families through rare founder events and then expands rapidly within their descendant lineages, producing punctuated adoption dynamics that likely induce distinct, transfer-limited scaling behavior for multimodal capabilities.

Summary

Main Finding

Multimodality in open LLM families typically appears via rare founder events (new VLM roots) and then expands rapidly within those VLM lineages. Direct lineage-level conversion from text-only checkpoints into vision–language models (VLMs) is exceptionally rare, so multimodal capability diffusion across families is punctuated and transfer-limited rather than gradual and pervasive.

Key Points

Dataset and scope: analysis uses the ModelBiome AI Ecosystem snapshot of Hugging Face (July 2025): 1.86 × 10^6 public models and 3.02 × 10^6 directed lineage relations.
Timing:
- Cross-modal tasks (text paired with image/audio/video) were common ecosystem-wide well before they became common inside major Transformer-based LLM families.
- Within major open LLM families, multimodality was rare through 2023 and most of 2024, then rose sharply across 2024–2025, driven mainly by image–text (vision–language) tasks.
- First VLM variants typically lag first text-generation family releases by months: e.g., ~1 month for Gemma, >1 year for several families, ~26 months for GLM.
Lineage-conditioned transition rates (conditioned on recorded parent→child relations):
- Fine-tuning: P(child = VLM | parent = text-generation) = 50/22,928 = 0.218% (95% Wilson CI 0.165–0.287).
- Merge: 12/11,594 = 0.104% (0.059–0.181).
- Quantization: 14/10,487 = 0.133% (0.080–0.224).
- Adapter channel sparse (1/97; unstable).
- Broader cross-modal (text with image/audio/video) changes only marginally increase fine-tune rate to 0.236%; ~93% of these are image–text.
Persistence within VLM lineages:
- 94.5% of VLM-child fine-tuning edges come from VLM parents (1005/1063).
- Conditional retention: 1005/1526 = 65.9% of fine-tune edges from VLM parents yield VLM children.
Founder structure:
- Roughly 60% of VLM releases are roots with no recorded parent.
- High concentration: the single top parent (naver-clova-ix/donut-base) accounts for 28.21% of recorded VLM→VLM fine-tune edges; top-3 parents account for 48.9%.
- Temporal pattern: rapid amplification from a small founder set followed by diversification (measured via top-1/top-3 shares and effective number of founders Neff = 1/HHI).
Time-resolved dynamics:
- Text→VLM fine-tune rate remains near zero most months, with episodic spikes (largest in Nov 2024: 10/1,061 = 0.943%); no sustained monotonic increase.
- VLM→VLM retention remains high across time, driving within-family VLM growth.

Data & Methods

Data source: ModelBiome AI Ecosystem dataset (Hugging Face snapshot, July 2025) linking model metadata, pipeline/task tags, semi-structured model cards, and recorded lineage edges.
Units and selection:
- Focus on Transformer-based models (excluding diffusion pipelines where noted).
- Monthly binning uses child model createdAt timestamps (Hugging Face createdAt available reliably from 2 Mar 2022 onward).
- Exclude months with low volume (e.g., n < 300) for stable estimates.
Task labeling:
- Pipeline_tag mappings used to identify text-generation, image-to-text, image+text-to-text (VLM), and other cross-modal tasks.
Lineage channels analyzed separately: finetune_parent, merge, quantize, adapter.
Metrics and inference:
- Conditional probabilities P(child task | parent task, relation) computed per month and aggregated.
- Uncertainty via 95% Wilson score intervals for binomial proportions.
- Founder concentration: counts of VLM→VLM finetune_parent edges by parent model_id; top-k shares and Neff = 1/HHI (effective number of parents) computed over monthly parent distributions.
Limitations acknowledged by authors:
- Task tags and model cards are noisy and heterogeneous; pipeline_tag coverage and quality vary.
- Recorded lineage is incomplete (many models have no parent recorded), so “roots” include both true new creations and cases with missing parent metadata.
- Lineage relations capture only what is recorded on Hugging Face and may miss external integration/engineering work not reflected in parent fields.

Implications for AI Economics

Diffusion dynamics and market structure:
- Founder-driven, punctuated adoption implies strong path dependence and concentration: early VLM founders can capture outsized derivative activity (network effects around reusable VLM checkpoints).
- Such concentration can produce winner-take-most dynamics within multimodal segments, affecting competition, pricing for services/tools, and bargaining power of founder checkpoint maintainers.
Barriers to entry and switching costs:
- Multimodality requires extra engineering (data pipelines, interface adapters, evaluation tooling). These integration costs act as barriers, slowing cross-family diffusion of multimodal capabilities and raising switching costs for adopters who need turnkey VLM stacks.
Value of standardization and tooling:
- Reducing integration friction (standardized vision-language interfaces, turnkey adapters, reliable adapter/LoRA toolchains, quantization-aware VLM workflows) would lower the cost of creating VLM descendants from text-only backbones, increasing lineage transfer rates and accelerating diffusion.
- Investments in such standards can shift the ecosystem from founder-dominated dynamics toward more incremental adoption—changing returns to R&D and the structure of complementarities between vision and language component providers.
Implications for investment and R&D strategy:
- Building or acquiring VLM “founders” (or taking early positions on promising founder checkpoints) may yield outsized returns via downstream derivatives.
- For firms focused on text-only model improvement, private returns into multimodal markets are limited unless accompanying integration work is performed; thus, strategic allocation of engineering resources matters.
Policy and platform considerations:
- Platform-level metadata completeness and lineage transparency matter for understanding ecosystem dynamics and for competition/regulation assessments; missing parent links can mask true diffusion paths.
- Open tooling and interoperable interfaces could democratize entry into multimodal markets; conversely, proprietary toolchains could entrench founders.
Empirical follow-ups suggested (economic research agenda):
- Quantify costs (engineering hours, dataset assembly) required to convert a text-only backbone into a VLM founder.
- Measure downstream economic value (downloads, citations, forks, commercial usage) accruing to founder checkpoints versus derived VLMs.
- Study how standardization initiatives (e.g., adapter libraries, interface specs) change measured lineage transition rates and concentration over time.
- Link lineage-derived measures to market outcomes: monetization, contributor incentives, and platform governance.

Summary takeaway: multimodal capabilities in open LLM families do not diffuse steadily from text-only improvements; they appear in punctuated founder events and then proliferate within those lineages. From an AI-economics perspective, this creates concentrated returns to founders, high integration frictions that act as barriers, and a clear leverage point—standardization and tooling—that could materially alter diffusion and market structure.

Assessment

Paper Typedescriptive Evidence Strengthmedium — The paper analyzes a very large dataset (>1.8M Hugging Face model entries) and documents clear descriptive patterns (timing, lineage, transition rates) that are robust to sample size; however, conclusions are limited by self-reported metadata, incomplete/missing lineage fields, and the observational/descriptive nature precludes causal claims about why multimodality emerges or spreads. Methods Rigormedium — The authors use systematic, reproducible analyses (time-series of modality tags, parent-child lineage-conditioned transition rates, founder-concentration metrics) on a comprehensive public metadata snapshot, which is appropriate for the descriptive questions; but rigor is constrained by potential measurement error (tagging inconsistencies), unobserved parentage, selection bias to Hugging Face/open models, and limited robustness checks or formal statistical tests reported in the summary. SampleModelBiome AI Ecosystem dataset of Hugging Face model metadata including recorded lineage fields and modality tags, covering over 1.8 million model entries and recorded parent-to-child relations across major open LLM families, with temporal coverage through parts of 2024–2025; analyses focus on classifying models as text-generation vs vision-language (VLM) and computing lineage-conditioned transition rates, first-release lags within named families, and founder/concentration statistics. Themesadoption innovation GeneralizabilityRestricted to Hugging Face / open-model ecosystem — excludes proprietary and closed-source models (e.g., large incumbent lab releases) which may follow different dynamics, Relies on self-reported metadata and lineage fields that are often missing or noisy, so observed parent-child edges understate true ancestry, Modality and family tags may be inconsistent across authors and over time, producing classification error, Temporal window through early/mid 2025 may miss later diffusion dynamics or delayed adoption in some families, Dynamics in open-source/model-hub communities may not generalize to enterprise adoption or commercial productization pathways

Claims (9)

Claim	Direction	Outcome	Confidence & Evidence	Details
Using the ModelBiome AI Ecosystem dataset of Hugging Face model metadata and recorded lineage fields (>1.8x10^6 model entries). Other	null_result	dataset_size_and_source	Reading fidelity high Study strength high	n=1800000 >1.8x10^6 model entries 0.3
Cross-modal tasks are widespread in the broader ecosystem well before they become common within major open LLM families. Adoption Rate	positive	timing_and_prevalence_of_cross-modal_tasks	Reading fidelity high Study strength medium	not reported 0.18
Within major open LLM families, multimodality remains rare through 2023 and most of 2024, then increases sharply in 2024-2025 and is dominated by image-text vision-language tasks. Adoption Rate	positive	prevalence_of_multimodality_over_time	Reading fidelity high Study strength medium	not reported 0.18
Across major families, the first vision-language model (VLM) variants typically appear months after the first text-generation releases, with lags ranging from ~1 month (Gemma) to more than a year for several families and ~26 months for GLM. Adoption Rate	null_result	lag_between_text_generation_and_first_VLM	Reading fidelity high Study strength medium	~1 month to ~26 months 0.18
Lineage-conditioned transition rates show weak cross-type transfer: among fine-tuning edges from text-generation parents, only 0.218% yield VLM descendants. Adoption Rate	negative	proportion_of_fine-tuning_edges_producing_VLM_from_text_parents	Reading fidelity high Study strength medium	0.218% 0.18
Multimodality expands primarily within existing VLM lineages: 94.5% of VLM-child fine-tuning edges originate from VLM parents, versus 4.7% from text-generation parents. Adoption Rate	positive	origin_of_VLM_children_by_parent_type	Reading fidelity high Study strength medium	94.5% from VLM parents; 4.7% from text-generation parents 0.18
At the model level, most VLM releases appear as new roots without recorded parents (~60%), while the remainder are predominantly VLM-derived. Adoption Rate	positive	proportion_of_VLM_releases_without_recorded_parents	Reading fidelity high Study strength medium	~60% 0.18
Founder concentration analyses indicate rapid within-lineage amplification followed by diversification. Adoption Rate	positive	within-lineage_amplification_and_subsequent_diversification	Reading fidelity medium Study strength medium	not reported 0.11
Multimodality enters open LLM families through rare founder events and then expands rapidly within their descendant lineages, producing punctuated adoption dynamics that likely induce distinct, transfer-limited scaling behavior for multimodal capabilities. Adoption Rate	positive	mode_of_entry_and_subsequent_expansion_of_multimodality	Reading fidelity high Study strength speculative	not reported 0.03