Decentralized ETS Counters for Better Scalability

August 03, 2021 · by Kjell Winblad .

A shared Erlang Term Storage (ETS) table is often an excellent place to store data that is updated and read from multiple Erlang processes frequently. ETS provides key-value stores to Erlang processes. When the write_concurrency option is activated, ETS tables use fine-grained locking internally. Therefore, a scenario where multiple processes insert and remove different items in an ETS table should scale well with the number of utilized cores. Unfortunately, in practice, the scalability for such scenarios is not yet perfect. However, the ETS table option decentralized_counters (introduced in Erlang/OTP 22 for ordered_set tables and in Erlang/OTP 23 for the other table types) has made the scalability much better. A table with decentralized_counters activated uses decentralized counters instead of centralized counters to track the number of items in the table and the memory consumption. Unfortunately, tables with decentralized_counters activated will have slow operations to get the table size and memory usage (ets:info(Table, size) and ets:info(Table, memory)), so whether it is beneficial to turn decentralized_counters on or off depends on your use case. This blog post will give you a better understanding of when one should activate the decentralized_counters option and how the decentralized counters work.

Read More

A few notes on message passing

March 19, 2021 · by John Högberg .

Message passing has always been central to Erlang, and while reasonably well-documented we’ve avoided going into too much detail to give us more freedom when implementing it. There’s nothing preventing us from describing it in a blog post though, so let’s have a closer look!

Read More

The Road to the JIT

December 01, 2020 · by Björn Gustavsson .

As long as Erlang has existed, there has always been the need and the ambition to make it faster. This blog post is a history lesson that outlines the major Erlang implementations and attempts to improve the performance of Erlang.

Read More

A first look at the JIT

November 03, 2020 · by John Högberg .

Now that we’ve had a look at BEAM and the interpreter we’re going to explore one of the most exciting additions in OTP 24: the just-in-time compiler, or “JIT” for short.

Read More

A brief introduction to BEAM

October 20, 2020 · by John Högberg .

This post is a brief primer on BEAM, the virtual machine that executes user code in the Erlang Runtime System (ERTS). It’s intended to help those new to BEAM follow an upcoming series of posts about the JIT in OTP 24, leaving implementation details for later.

Read More

The New Scalable ETS ordered_set

August 19, 2020 · by Kjell Winblad .

The scalability of ETS tables of type ordered_set with the write_concurrency option is substantially better in Erlang/OTP 22 than earlier releases. In some extreme cases, you can expect more than 100 times better throughput in Erlang/OTP 22 compared to Erlang/OTP 21. The cause of this improvement is a new data structure called the contention adapting search tree (CA tree for short). This blog post will give you insights into how the CA tree works and show you benchmark results comparing the performance of ETS ordered_set tables in OTP 21 and OTP 22.

Read More

OTP 23 Highlights

May 13, 2020 · by Kenneth Lundin .

OTP 23 has just been released (May 13:th 2020). It has been a long process with three release candidates in February, March and April before the final release. We are very thankful for the feedback we have got regarding the release candidates, which has revealed some bugs and flaws that our internal testing did not find.

Read More

OTP 22 Highlights

May 13, 2019 · by Lukas Larsson .

OTP 22 has just been released. It has been a long process with three release candidates before the final release. We decided this year to try to get one month more testing of the major release and I think that the extra time has paid off. We’ve received many bug reports from the community about large and small bugs that our internal tests did not find.

Read More

ETS oddity

January 07, 2019 · by Lukas Larsson .

When working with the implementation of the new scalable ordered_set we came across a strangeness with the guarantees when iterating over a table while inserting elements in parallel.

Read More

Retiring old performance pitfalls

November 07, 2018 · by John Högberg .

Erlang/OTP 22 will bring many performance improvements to the table, but most of them have a broad impact and don’t affect the way you write efficient code. In this post I’d like to highlight a few things that used to be surprisingly slow but no longer need to be avoided.

Read More

TLS logging improvements in OTP 22

October 05, 2018 · by Péter Dimitrov .

Erlang/OTP 22 will be an important release for the ssl application. We are working on several new features and improvements such as support for TLS 1.3, some of those are already on the master branch. This blog post presents the new ssl debug logging built on the new logger API.

Read More

Optimization Traps and Pitfalls

August 24, 2018 · by Björn Gustavsson .

Back after the summer holidays, this blog will now change tracks and start a series of blog posts about Static Single Assignment (SSA). This first installment will set the scene for the posts that follow by looking at the traps and pitfalls one can fall into when trying to optimize BEAM assembly code.

Read More

A Brief History of the BEAM Compiler

June 18, 2018 · by Björn Gustavsson .

This blog post is a brief history lesson about the Erlang compiler for the BEAM machine. To provide some context, there will first be a quick look at the abstract machines for Erlang.

Read More

Interpreter optimization

June 11, 2018 · by Lukas Larsson .

The BEAM interpreter in erts has been completely re-written in OTP 21. Most of the instructions have remained the same, but the perl scripts used to generate the C code have a new implementation. This blog post will look at some of the optimizations that were possible because of those changes.

Read More

Core Erlang Wrap Up

May 30, 2018 · by Björn Gustavsson .

This blog post wraps up the exploration of Core Erlang started in the previous two blog posts. The remaining default Core Erlang passes are described, followed by a look at how Core Erlang is represented internally in the compiler.

Read More

Core Erlang Optimizations

May 18, 2018 · by Björn Gustavsson .

This blog post continues the exploration of Core Erlang by looking at some optimizations done by the sys_core_fold compiler pass. The Core Erlang language was introduced in the previous blog post.

Read More

Core Erlang by Example

May 07, 2018 · by Björn Gustavsson .

This blog post is the first about the Core Erlang format. In this blog post, we introduce the Core Erlang format through examples that compare Erlang code to the corresponding Core Erlang code.

Read More

My OTP 21 Highlights

May 02, 2018 · by Lukas Larsson .

OTP-21 Release Candidate 1 has just been released. I thought that I would go through the changes that I am the most excited about. Most likely this will mostly mean features in erts and the core libraries as those are the changes that I am the most familiar with.

Read More

Memory instrumentation in OTP 21

May 02, 2018 · by John Högberg .

The memory instrumentation module was rewritten for Erlang/OTP 21 to make it easier to use. In this post I’ll describe the rationale behind the new features and how to make use of them.

Read More

Exploring the Compiler Using the 'time' Option

April 19, 2018 · by Björn Gustavsson .

This is the first of a series of blog posts about the compiler. There will be blog posts about how the compiler works now, how it might work in the future, and some historical notes to explain why some things are what they are. In this blog post I will talk about one of the most useful options for exploring the compiler, namely the time option.

Read More

I/O polling options in OTP 21

April 11, 2018 · by Lukas Larsson .

Erlang/OTP 21 will introduce a completely new IO polling implementation. This new implementation comes with a new set of tuneable parameters that can be used to get the most out of your system. This blog post describes the parameters and attempts to describe what they should be used for.

Read More