🗄️Git Large File Storage

NEW

Also known as: Git LFS

An extension that manages large files by storing pointers in Git and content on external storage.

DESCRIPTION

Git LFS replaces large files with lightweight pointers in the Git history while storing actual content on a remote server. It keeps repositories small and efficient, improving performance for teams handling datasets, media, or model artifacts. Integration with major platforms simplifies setup and supports CI/CD workflows.

ALTERNATIVE DEFINITIONS

Git LFS

Git LFS replaces large files such as audio samples, videos, datasets, and graphics with text pointers inside Git, while storing the file contents on a remote server.

Highlights pointer replacement and remote storage for large binaries.

Source

GitHub Docs

Git Large File Storage (Git LFS) replaces large files with pointers and stores the file contents in a separate Git LFS server.

Describes server separation and pointer mechanism clearly.

Source

RELATED TERMS

Git Artifact Repository Binary File Model Artifact S3 GitHub GitLab Bitbucket CI-CD

COMPETING TERMS

git-annex Data Version Control

RELATIONS

• Broader: File Storage

KNOWLEDGE AREAS

Software Engineering Platform & Tooling Data Engineering MLOps

TARGET AUDIENCE

Software Developer Data Engineer MLOps Engineer AI Engineer DevOps Engineer

EXTERNAL SOURCES

• Wikipedia - Git LFS
• Git LFS - Official Git LFS site and docs
• GitHub - Platform guide for Git LFS
• Atlassian - Tutorial on Git LFS usage

TOP QUESTIONS

Why use Git LFS for large files?

It keeps repositories lightweight and fast by storing binaries externally, improving clone and fetch times.

Category: Relevancy

How does Git LFS work?

It replaces large files with pointer files in Git and serves content from an LFS server during checkout.

Category: Example

What’s a common mistake with Git LFS?

Not configuring tracked patterns early, leading to bloated history and complicated cleanups.

Category: Mistakes

What are alternatives to Git LFS?

git-annex, dedicated artifact repositories, or DVC for data pipelines and versioning.

Category: Alternatives

What risks come with Git LFS?

Server limits, bandwidth usage, and storage costs can surprise teams without monitoring.

Category: Risk

How to manage Git LFS in CI/CD?

Cache artifacts, prefetch LFS, and use selective checkout to optimize build times.

Category: Management

Is Git LFS suitable for datasets in MLOps?

Yes, for moderate-sized data; for very large datasets, consider data lakes or specialized storage.

Category: Opinion

How do I start with Git LFS?

Install Git LFS, run ‘git lfs install’, track patterns with ‘git lfs track’, and commit pointer files.

Category: Requirements

Can Git LFS be used on-prem?

Yes, with self-hosted LFS servers or Git platforms that support LFS.

Category: Example