On twitter today I tweeted the following:

https://twitter.com/ashsmithco/status/1177996134322708490

Yep. I found a way of reducing a 799mb pub/static directory in Magento to just 80mb. I was a little shocked.

In this example, we have 7 frontend themes and 4 locales. So that's 28 directories containing a lot of duplicated files. Additionally, we have 2 locales used on the adminhtml theme.

After running a few tests, I can confirm it works, and without any pitfuls that I can foresee!

All you'll need to do is to install rdfind, and run rdfind -makehardlinks true pub/static. Integrate this with your CI/CD pipeline, and you'll be saving plenty of disk space, but also network transfer too.

This has massive value when you're trying to keep Docker image sizes down. My Magento image went from 509MB to 245MB (when compressed). That's reduced its size by over half!

How does this sourcery work!?

rdfind will search all files in the given path, check their contents, and find duplicates, and then with the -makehardlinks true flag set it will replace duplicates with hard links.

It uses an algorithm to do it effectively, and you can read more here: https://github.com/pauldreik/rdfind

So what are hard links?

Wikipedia says the following:

In computing, a hard link is a directory entry that associates a name with a file on a file system. All directory-based file systems must have at least one hard link giving the original name for each file. The term “hard link” is usually only used in file systems that allow more than one hard link for the same file.

Effectively what is happening is we have multiple files that have a 'hard link' to the same inode, or same file that is written to the disk.

Hard links have some limitations:

• You can't use them on directories on most operating systems.
• You can't use them across volumes.

If you'd prefer to use a soft link (or symlink) rdfind also supports this.

With great power...

Use this cautiously, and sparingly. In the case of Magento it makes sense to be using this on the pub/static directory for a production mode website after static content has been compiled/deployed. These files shouldn't be written to again, and your next deployment you'll be recreating those directories anyway.

Plus if you're running bin/magento setup:static-content:deploy as part of your CI process, you'll always have a clean filesystem to run rdfind in.