Meta’s renewed commitment to jemalloc(engineering.fb.com)
245 points by hahahacorn 3 hours ago | 104 comments
- adsharma 1 hour ago> We plan to deliver improvements to [..] purging mechanisms
During my time at Facebook, I maintained a bunch of kernel patches to improve jemalloc purging mechanisms. It wasn't popular in the kernel or the security community, but it was more efficient on benchmarks for sure.
Many programs run multiple threads, allocate in one and free in the other. Jemalloc's primary mechanism used to be: madvise the page back to the kernel and then have it allocate it in another thread's pool.
One problem: this involves zero'ing memory, which has an impact on cache locality and over all app performance. It's completely unnecessary if the page is being recirculated within the same security domain.
The problem was getting everyone to agree on what that security domain is, even if the mechanism was opt-in.
- bmenrigh 3 hours agoI recently started using Microsoft's mimalloc (via an LD_PRELOAD) to better use huge (1 GB) pages in a memory intensive program. The performance gains are significant (around 20%). It feels rather strange using an open source MS library for performance on my Linux system.
There needs to be more competition in the malloc space. Between various huge page sizes and transparent huge pages, there are a lot of gains to be had over what you get from a default GNU libc.
[-]- skavi 2 hours agoWe evaluated a few allocators for some of our Linux apps and found (modern) tcmalloc to consistently win in time and space. Our applications are primarily written in Rust and the allocators were linked in statically (except for glibc). Unfortunately I didn't capture much context on the allocation patterns. I think in general the apps allocate and deallocate at a higher rate than most Rust apps (or more than I'd like at least).
Our results from July 2025:
rows are <allocator>: <RSS>, <time spent for allocator operations>
tcmalloc was from github.com/google/tcmalloc/tree/24b3f29.app1: glibc: 215,580 KB, 133 ms mimalloc 2.1.7: 144,092 KB, 91 ms mimalloc 2.2.4: 173,240 KB, 280 ms tcmalloc: 138,496 KB, 96 ms jemalloc: 147,408 KB, 92 ms app2, bench1 glibc: 1,165,000 KB, 1.4 s mimalloc 2.1.7: 1,072,000 KB, 5.1 s mimalloc 2.2.4: tcmalloc: 1,023,000 KB, 530 ms app2, bench2 glibc: 1,190,224 KB, 1.5 s mimalloc 2.1.7: 1,128,328 KB, 5.3 s mimalloc 2.2.4: 1,657,600 KB, 3.7 s tcmalloc: 1,045,968 KB, 640 ms jemalloc: 1,210,000 KB, 1.1 s app3 glibc: 284,616 KB, 440 ms mimalloc 2.1.7: 246,216 KB, 250 ms mimalloc 2.2.4: 325,184 KB, 290 ms tcmalloc: 178,688 KB, 200 ms jemalloc: 264,688 KB, 230 msi don't recall which jemalloc was tested.
[-]- codexon 6 minutes agoThis is similar to what I experienced when I tested mimalloc many years ago. If it was faster, it wasn't faster by much, and had pretty bad worst cases.
- hedora 2 hours agoI’m surprised (unless they replaced the core tcmalloc algorithm but kept the name).
tcmalloc (thread caching malloc) assumes memory allocations have good thread locality. This is often a double win (less false sharing of cache lines, and most allocations hit thread-local data structures in the allocator).
Multithreaded async systems destroy that locality, so it constantly has to run through the exception case: A allocated a buffer, went async, the request wakes up on thread B, which frees the buffer, and has to synchronize with A to give it back.
Are you using async rust, or sync rust?
[-]- skavi 1 hour agomodern tcmalloc uses per CPU caches via rseq [0]. We use async rust with multithreaded tokio executors (sometimes multiple in the same application). so relatively high thread counts.
[0]: https://github.com/google/tcmalloc/blob/master/docs/design.m...
[-]- usrnm 15 minutes agoHow do you control which CPU your task resumes on? If you don't then it's still the same problem described above, no?
- ComputerGuru 2 hours agoThat’s a considerable regression for mimalloc between 2.1 and 2.2 – did you track it down or report it upstream?
Edit: I see mimalloc v3 is out – I missed that! That probably moots this discussion altogether.
[-]- skavi 1 hour agonope.
- pjmlp 3 hours agoIf you go into Dr Dobbs, The C/C++ User's Journal and BYTE digital archives, there will be ads of companies whose product was basically special cased memory allocator.
Even toolchains like Turbo Pascal for MS-DOS, had an API to customise the memory allocator.
The one size fits all was never a solution.
- adgjlsfhk1 3 hours agoOne of the best parts about GC languages is they tend to have much more efficient allocation/freeing because the cost is much more lumped together so it shows up better in a profile.[-]
- pjmlp 3 hours agoAgreed, however there is also a reason why the best ones also pack multiple GC algorithms, like in Java and .NET, because one approach doesn't fit all workloads.[-]
- nevdka 2 hours agoThen there’s perl, which doesn’t free at all.[-]
- hedora 2 hours agoPerl frees memory. It uses refcounting, so you need to break heap cycles or it will leak.
(99% of the time, I find this less problematic than Java’s approach, fwiw).
- cermicelli 2 hours agoFreedom is overrated... :P
- NooneAtAll3 2 hours agodoesn't java also?
I heard that was a common complaint for minecraft
[-]- xxs 2 hours agoWhat do you mean - if Java returns memory to the OS? Which one - Java heap of the malloc/free by the JVM?[-]
- cogman10 1 hour agoJava is pretty greedy with the memory it claims. Especially historically it was pretty hard to get the JVM to release memory back to the OS.
To an outsider, that looks like the JVM heap just steadily growing, which is easy to mistake for a memory leak.
[-]- k_roy 1 hour ago> Especially historically it was pretty hard to get the JVM to release memory back to the OS.
This feels like a huge understatement. I still have some PTSD around when I did Java professionally between like 2005 and 2014.
The early part of that was particularly horrible.
- adgjlsfhk1 41 minutes agoThis only really ends up being a problem on windows. On systems with proper virtual memory setups, the cost of unused memory is very low (since the the OS can just page it out)
- xxs 1 hour agoJava has a quite strict max heap setting, it's very uncommon to let it allocate up to 25% of the system memory (the default). It won't grow past that point, though.
Baring bugs/native leaks - Java has a very predictable memory allocation.
- bluGill 2 hours agoWhen it works. Many programs in GC language end up fighting the GC by allocating a large buffer and managing it by hand anyway because when performance counts you can't have allocation time in there at all. (you see this in C all the time as well)[-]
- cogman10 1 hour agoThat's generally a bad idea. Not always, but generally.
It was a better idea when Java had the old mark and sweep collector. However, with the generational collectors (which are all Java collectors now. except for epsilon) it's more problematic. Reusing buffers and objects in those buffers will pretty much guarantees that buffer ends up in oldgen. That means to clear it out, the VM has to do more expensive collections.
The actual allocation time for most of Java's collectors is almost 0, it's a capacity check and a pointer bump in most circumstances. Giving the JVM more memory will generally solve issues with memory pressure and GC times. That's (generally) a better solution to performance problems vs doing the large buffer.
Now, that said, there certainly have been times where allocation pressure is a major problem and removing the allocation is the solution. In particular, I've found boxing to often be a major cause of performance problems.
[-]- CyberDildonics 23 minutes agoIf people didn't need to do it, they wouldn't generally do it. Not always, but generally.[-]
- cogman10 17 minutes agoPeople do stuff they shouldn't all the time.
For example, some code I had to clean up pretty early on in my career was a dev, for unknown reasons, reinventing the `ArrayList` and then using that invention as a set (doing deduplication by iterating over the elements and checking for duplicates). It was done in the name of performance, but it was never a slow part of the code. I replaced the whole thing with a `HashSet` and saved ~300 loc as a result.
This individual did that sort of stuff all over the code base.
- CyberDildonics 23 minutes agoAny extra throughput is far overshadowed by trying to control pauses and too much heap allocations happening because too much gets put on the heap. For anything interactive the options are usually fighting the gc or avoiding gc.
- m463 1 hour agoI remember in the early days of web services, using the apache portable runtime, specifically memory pools.
If you got a web request, you could allocate a memory pool for it, then you would do all your memory allocations from that pool. And when your web request ended - either cleanly or with a hundred different kinds of errors, you could just free the entire pool.
it was nice and made an impression on me.
I think the lowly malloc probably has lots of interesting ways of growing and changing.
[-]- jra_samba 44 minutes agoLook into talloc, used inside Samba (and other FLOSS projects like sssd). Exactly this.
- Dylan16807 37 minutes agoI think some operating system improvements could get people motivated to use huge pages a lot better. In particular make them less fragile on linux and make them not need admin rights on windows. The biggest factor causing problems there is that neither OS can swap a 2MB page. So someone needs to care enough to fix that.
- pocksuppet 2 hours agoIn many cases you can also do better than using malloc e.g. if you know you need a huge page, map a huge page directly with mmap
Yes, if you want to use huge pages with arbitrary alloc/free, then use a third-party malloc. If your alloc/free patterns are not arbitrary, you can do even better. We treat malloc as a magic black box but it's actually not very good.
- codexon 2 hours agoI've been using jemalloc for over 10 years and don't really see a need for it to be updated. It always holds up in benchmarks against any new flavor of the month malloc that comes out.
Last time I checked mimalloc which was admittedly a while ago, probably 5 years, it was noticebly worse and I saw a lot of people on their github issues agreeing with me so I just never looked at it again.
[-]- adgjlsfhk1 2 hours agoMimalloc v3 has just come out (about a month ago) and is a significant improvement over both v2 and v1 (what you likely last tested)
- hrmtst93837 2 hours agoBenchmarks age fast. Treating a ten-year-old allocator as done just because it still wins old tests is tempting fate, since distros, glibc, kernel VM behavior, and high-core alloc patterns keep moving and the failures usually show up as weird regressions in production, not as a clean loss on someone's benchmark chart.[-]
- codexon 2 hours agoIt still beat mimalloc when I checked 4-5 years ago.[-]
- imp0cat 2 hours agoYou really need to benchmark your workloads, ideally with the "big 3" (jemalloc, tcmalloc, mimalloc). They all have their strengths and weaknesses.
Jemalloc can usually keep the smallest memory footprint, followed by tcmalloc.
Mimalloc can really speed things up sometimes.
As usually, YMMV.
[-]- codexon 2 hours agoI've benchmarked them every few years, they never seem to differ by more than a few percent, and jemalloc seems to fragment and leak the least for processes running for months.
Mimalloc made the claim that they were the fastest/best when they released and that didn't hold up to real world testing, so I am not inclined to trust it now.
[-]- ComputerGuru 1 hour ago> Mimalloc made the claim that they were the fastest/best when they released and that didn't hold up to real world testing
That’s… ahistorical, at least so far as I remember. It wasn’t marketed as either of those; it was marketed as small/simple/consistent with an opt-in high-severity mode, and then its performance bore out as a result of the first set of target features/design goals. It was mainly pushed as easy to adopt, easy to use, easy to statically link, etc.
[-]- codexon 12 minutes ago> It was mainly pushed as easy to adopt, easy to use, easy to statically link, etc.
That is true of basically every single malloc replacement out there, that is not a uniquely defining feature.
- HackerThemAll 27 minutes agoLook up the numbers in other comments above. When it comes to performance, the Google's tcmalloc is unconquered.
- IshKebab 2 hours agoI feel like the real thing that needs to change is we need a more expressive allocation interface than just malloc/realloc. I'm sure that memory allocators could do a significantly better job if they had more information about what the program was intending to do.[-]
- liuliu 2 hours agoThere are, look no further than jemalloc API surface itself:
https://jemalloc.net/jemalloc.3.html
One thing to call out: sdallocx integrates well with C++'s sized delete semantics: https://isocpp.org/files/papers/n3778.html
[-]- hedora 1 hour agoYou can also play tricks with inlining and constant propagation in C (especially on the malloc path, where the ground-truth allocation size is usually statically known).
- anthk 2 hours agoI used mimalloc to run zenlisp under OpenBSD as it would clash with the paranoid malloc of base.
- jeffbee 2 hours agoJust out of curiosity are you getting 1GB huge pages on Xeon or some other platform? I always thought this class of page is the hardest to exploit, considering that the machine only has, if I recall correctly, one TLB slot for those.[-]
- bmenrigh 2 hours agoModern x86_64 has supported multiple page sizes for a long time. I'm on commodity Zen 5 hardware (9900X) with 128 GiB of RAM. Linux will still use a base page size of 4kb but also supports both 2 MiB and 1 GiB huge pages. You can pass something like `default_hugepagesz=2M hugepagesz=1G hugepages=16` to your kernel on boot to use 2 MiB pages but reserve 16 1 GiB pages for later use.
The nice thing about mimalloc is that there are a ton of configurable knobs available via env vars. I'm able to hand those 16 1 GiB pages to the program at launch via `MIMALLOC_RESERVE_HUGE_OS_PAGES=16`.
EDIT: after re-reading your comment a few times, I apologize if you already knew this (which it sounds like you did).
[-]- jeffbee 1 hour agoRight but on Intel the 1G page size has historically been the odd one. For example Skylake-X has 1536 L2 shared TLB entries for either 4K or 2M pages, but it only has 16 entries that can be used for 1G pages. It wasn't unified until Cascade Lake. But Skylake-like Xeon is still incredibly common in the cloud so it's hard to target the later ones.[-]
- Dylan16807 23 minutes agoSo for any process that's using less than 16GB, it's a significant performance boost. And most processes using more RAM, but not splitting accesses across more than 16 zones in rapid succession, will also see a performance boost.
My old Intel CPU only has 4 slots for 1GB pages, and that was enough to get me about a 20% performance boost on Factorio. (I think a couple percent might have been allocator change but the boost from forcing huge pages was very significant)
- sylware 3 hours agoIf there is so much performance difference among generic allocators, it means you need semantic optimized allocators (unless performance is actually not that much important in the end).[-]
- codexon 9 minutes agoAgreed mostly. Going from standard library to something like jemalloc or tcmalloc will give you around 5-10% wins which can be significant, but the difference between those generic allocators seem small. I just made a slab allocator recently for a custom data type and got speedups of 100% over malloc.
- Cloudef 2 hours agoYou are not wrong and this is indeed what zig is trying to push by making all std functions that allocate take a allocator parameter.
- dang 3 hours agoRelated. Others?
Jemalloc Postmortem - https://news.ycombinator.com/item?id=44264958 - June 2025 (233 comments)
Jemalloc Repositories Are Archived - https://news.ycombinator.com/item?id=44161128 - June 2025 (7 comments)
- bfgeek 3 hours agoOne has to wonder if this due to the global memory shortage. ("Oh - changing our memory allocator to be more efficient will yield $XXM dollar savings over the next year").[-]
- bluGill 1 hour agoFacebook had talks already years ago (10+) - nobody was allowed to share real numbers, but several facebook employed where allowed to share that the company has measured savings from optimizations. Reading between the lines, a 0.1% efficiency improvement to some parts of Facebook would save them $100,000 a month (again real numbers were never publicly shared so there is a range - it can't be less than $20,000), and so they had teams of people whose job it was to find those improvements.
Most of the savings seemed to come from HVAC costs, followed by buying less computers and in turn less data centers. I'm sure these days saving memory is also a big deal but it doesn't seem to have been then.
The above was already the case 10 years ago, so LLMs are at most another factor added on.
[-]- sethhochberg 42 seconds agoI don't have many regrets about having spent my career in (relatively) tiny companies by comparison, but it sure does sound fun to be on the other side for this kind of thing - the scale where micro-optimizations have macro impact.
- alex1138 32 minutes agoI've heard of some people getting banned from FB to save memory space? Surely that can't be the case but I swear I've seen something like that
- HackerThemAll 30 minutes ago> LLMs are at most another factor added on
At most... Think 10x rather than 0.1x or 1x.
- runevault 2 hours agoOn top of cost, they probably cannot get as much memory as they order in a timely fashion so offsetting that with greater efficiency matters right now.
- Nuzzerino 44 minutes agoWith the reputation of that company, one can wonder a lot of backstories that are even more depressing than a memory shortage.
- loeg 1 hour agoYeah, identifying single-digit millions of savings out of profiles is relatively common practice at Meta. It's ~easy to come up with a big number when the impact is scaled across a very large numbers of servers. There is a culture of measuring and documenting these quantified wins.
- foobarian 1 hour agoOooh maybe finally time for lovingly hand-optimized assembly to come back in fashion! (It probably has in AI workloads or so I daydream)
- augusto-moura 2 hours agoNot just shortage, any improvements to LLMs/electricity/servers memory footprint is becoming much more valuable as the time goes. If we can get 10% faster, you can easily get a lead in the LLM race. The incentives to transparently improving performance are tremendous
- mathisfun123 28 minutes ago> changing our memory allocator
they've been using jemalloc (and employing "je") since 2009.
- jshorty 60 minutes agoSurprised not to see any mention of the global memory supply shock. Would love to learn more about how that economic is shifting software priorities toward memory allocation for the first time in my (relatively young) career[-]
- twodave 35 minutes agoWhile it may seem directly related, it's just not. These things are worked on regardless of how cheap or expensive RAM is, because optimizing memory footprint pretty much always leads to fewer machines leased, which is a worthwhile goal even for smaller shops.
- refulgentis 32 minutes agoThere’s been shocks at hyperscaler scale, ex. this got yuge at Google for a couple years before ChatGPT
- jjuliano 1 hour agoI remember I was a senior lead softeng of a worldbank funded startup project, and have deployed Ruby with jemalloc in prod. There's a huge noticeable speed and memory efficiency. It did saved us a lot of AWS costs, compare to just using normal Ruby. This was 8 years ago, why haven't projects adopt it yet as de facto.
- joelsiks 1 hour agoOpening up strong with a gigantic merge of the stuff they've been working on in their own fork: https://github.com/jemalloc/jemalloc/pull/2863
- rishabhjajoriya 1 hour agoLarge engineering orgs often underestimate how much CI pipelines amplify performance issues. Even small inefficiencies multiply when builds run hundreds of times a day.
- Nuzzerino 36 minutes ago> Building a software system is a lot like building a skyscraper: The product everyone sees is the top, but the part that keeps it from falling over is the foundation buried in the dirt and the scaffolding hidden from sight.
They should have just called it an ivory tower, as that's what they're building whenever they're not busy destroying democracy with OS Backdoor lobbyism or Cambridge Analytica shenanigans.
Edit: If every thread about any of Elon Musk's companies can contain at least 10 comments talking about Elon's purported crimes against humanity, threads about Zuckerberg's companies can contain at least 1 comment. Without reminders like this, stories like last week's might as well remain non-consequential.
- RegnisGnaw 3 hours agoIs there a concise timelime/history of this? I thought jemalloc was 100% open source, why is Meta in control of it?[-]
- masklinn 2 hours agoJason Evans (the creator of jemalloc) recounted the entire thing last year: https://jasone.github.io/2025/06/12/jemalloc-postmortem/[-]
- vintermann 2 hours ago"Were I to reengage, the first step would be at least hundreds of hours of refactoring to pay off accrued technical debt."
Facebook's coding AIs to the rescue, maybe? I wonder how good all these "agentic" AIs are at dreaded refactoring jobs like these.
[-]- xxs 2 hours agoRefactor doesn't mean just artificial puff-up jobs, it's very likely internal changes and reorganization (hence 100s of hours).
There are not many engineers capable of working on memory allocators, so adding more burden by agentic stuff is unlikely to produce anything of value.
- rvz 2 hours ago> Facebook's coding AIs to the rescue, maybe? I wonder how good all these "agentic" AIs are at dreaded refactoring jobs like these.
No.
This is something you shouldn't allow coding agents anywhere near, unless you have expert-level understanding required to maintain the project like the previous authors have done without an AI for years.
- echelon 2 hours agoIf you filter the commits to the past five years, four of the top six committers are Meta employees. The other two might be as well, it just doesn't say that on their Github / personal website.
- pram 2 hours agoI used jemalloc recently for ComfyUI/Wan and it’s literally magic. I’m surprised it doesn’t come that way by default.
- thatoneengineer 3 hours agoFirst impressions: LOL, the blunt commentary in the HN thread title compared to the PR-speak of the fb.com post.
Second thoughts: Actually the fb.com post is more transparent than I'd have predicted. Not bad at all. Of course it helps that they're delivering good news!
[-]- MBCook 2 hours agoIt’s still quite corporate-y, but other than the way of writing I agree it’s generally quite clear.
- starkparker 2 hours ago(wrong thread)[-]
- gcr 1 hour agoThe URL of this story seems to have changed to a Meta press release. What are you quoting?[-]
- starkparker 12 minutes agoSorry, misdirected the reply.
- nubinetwork 3 hours agoSomeone should tell Bryan Cantrill, he'd probably be ecstatic...
- carlos256 28 minutes agoIf you need to optimize the allocator you are doing it wrong.
- xxs 2 hours agoFew months back, some of the services switched to jemalloc for the Java VM. It took months (of memory dumps and tracing sys-calls) to blame the JVM, itself, for getting killed by the oom_killer.
Initially the idea was diagnostics, instead the the problem disappeared on its own.
[-] - markstos 3 hours agoHow is the original author making out in the new arrangement?[-]
- Aurornis 1 hour agoJason Evans worked for Facebook for almost two decades, starting in 2009 - https://jasone.github.io/2025/06/12/jemalloc-postmortem/
He's doing just fine. If you're looking for a story about a FAANG company not paying engineers well for their work, this isn't it.
- lobf 1 hour ago>We are committed to continuing to develop jemalloc development
From the Department of Redundancy Department.
- m3kw9 50 minutes agoAll the AI investment and their biggest news is commitment to Jemalloc
- flykespice 2 hours agoJemalloc also is used by android bionic libc library[-]
- tonfa 2 hours agoDoesn't it depend on vendors/customization? Default is https://llvm.org/docs/ScudoHardenedAllocator.html since Android 11 (2020).[-]
- flykespice 39 minutes agoI don't know, that is probably the case I guess?
I was recently debugging an app double-free segfault on my android 13 samsung galaxy A51 phone, and the internal stack trace pointed to jemalloc function calls (je_free).
- charcircuit 3 hours agoMeta never abandoned jemalloc. https://github.com/facebook/jemalloc remained public the entire time. It's my understanding that Jason Evans, the creator of jemalloc, had ownership over the jemalloc/jemalloc repo which is why that one stopped being updated after he left.[-]
- kstrauser 3 hours agoThe repo's availability isn't related to whether it's still maintained.[-]
- charcircuit 3 hours agoMeta still maintained it and actively pushed commits to it fixing bugs and adding improvements. From this blog post it sounds like they are increasing investment into it along with resurrecting the original repo. When the repo was archived Meta said that development on jemalloc would be focused towards Meta's own goals and needs as opposed to the larger ecosystem.[-]
- kstrauser 2 hours agoI'm not directly involved enough to dig into the details here, but facebook/jemalloc currently says:
> This branch is 71 commits ahead of and 70 commits behind jemalloc/jemalloc:dev.
It looks like both have been independently updated.
[-]- Xylakant 1 hour agoThis looks a lot as if the facebook/jemalloc repo inserted a single commit 70 commits ago and then rebased the changes in the original repo on top. Because the commit SHAs for the changes pulled in change you see this result.
- masklinn 2 hours agoThe team probably sync'd the two after unarchiving the original.
- fermentation 3 hours agoSeems like they’d want to wait to commit until after the layoffs, right?[-]
- OsrsNeedsf2P 2 hours agoI work in the space. This article would not have been published if the team responsible was on the chopping block[-]
- VoidWarranty 53 minutes ago[dead]
- kubb 1 hour agoIt's just one team with like 4 people. They can layoff a lot of staff from Metaverse.
- rgupta1833 3 hours ago[dead]
- lesscraft 2 hours ago[dead]
- oncallthrow 3 hours agoAnd the Oscar for most mealy-mouthed post of the year goes to…[-]
- dang 3 hours agoWe generally try to avoid corporate press releases for that reason, but is there a good third-party post to replace it with?
https://hn.algolia.com/?dateRange=all&page=0&prefix=true&sor...