I also filed this here. The error received and the reason why are different on
Ubuntu and Debian, so the solution might be similar or it might be different.
So, I am just trying to be complete / comprehensive.

https://bugs.launchpad.net/ubuntu/+source/redis/+bug/1890517

Matthew.

tags 967970 + moreinfo
thanks

Hi Matthew,
Ubuntu. So, for completeness, I can "reproduce" this in Debian:

Aug 06 12:42:10 tinycat systemd[1]: Starting Advanced key-value store...
Aug 06 12:42:10 tinycat redis-server[2682431]: *** FATAL CONFIG FILE ERROR (Redis 6.0.6) ***
Aug 06 12:42:10 tinycat redis-server[2682431]: Reading the configuration file, at line 1783
Aug 06 12:42:10 tinycat redis-server[2682431]: >>> 'activedefrag yes'
Aug 06 12:42:10 tinycat redis-server[2682431]: Active defragmentation cannot be enabled: it requires a Redis server compiled with a modified Jemalloc like the one shipped by default with the Redis source distribution
Aug 06 12:42:10 tinycat systemd[1]: redis-server.service: Control process exited, code=exited, status=1/FAILURE

I use the term "reproduce" in square quotes here because I am not
entirely sure why you are surprised by this error -- we deliberately
do not use the bundled version of jemalloc in Debian so we can use the
 system one as per the general policy about avoiding embedded code
copies.

The only solutions would appear to be either (a) revert to using the
bundled jemalloc version (which would require a stronger case at this
point), or (b) ask upstream to merge this fragmentation support into
upstream jemalloc.

What am I missing here? I might be missing something as I don't
completely understand your remarks about the JEMALLOC_FRAG_HINT
define. Indeed, the code that surrounds it in zmalloc.h appears to
perfectly describe the trade-off being made:

 73 /* We can enable the Redis defrag capabilities only if we are using Jemalloc
 74  * and the version used is our special version modified for Redis having
 75  * the ability to return per-allocation fragmentation hints. */
 76 #if defined(USE_JEMALLOC) && defined(JEMALLOC_FRAG_HINT)
 77 #define HAVE_DEFRAG
 78 #endif


Regards,

Thanks for confirming, I was able to get the same behavior when I tried it on
your latest Debian source package, hence why I felt reasonably comfortable it
applied to both distributions.
Debian (or Ubuntu derivatives) since about 1997 and I am aware of and strongly
support the general policy.

But at the same time, I wanted to be sure the community was made aware of it
if they were setting up the service and expected the feature to work, and
advise that at least a base config file comment about it could be a good idea
if everybody decides not to fix it.

I think this analysis is probably correct, depending on the estimated
importance of the feature.

If you could provide some advice, of the right way to temporarily disable the
USE_SYSTEM_JEMALLOC patch, for building a one-off Debian package of the
redis-server where defrag would be able to operate, probably that's good
enough so that somebody would see it in the public historical record if they
run into this later.

I think you got it right.

Sorry, I was pretty tired when I wrote it, as I found it late in a 16 hour day
of coding something else which had redis-server as a dependency, and took a
break to debug it and report it in an effort to give back to the community at
least a little bit, after I found it was apparently not seen before online in
my search engine results.

True, but that's a good few layers deep, so I was trying to give it a bit more
exposure for the next user, whoever that might be.

Thanks for your prompt reply and analysis, and for maintaining the package.

Matthew.

Hi Matthew,

Good idea. I'll add a comment to the config file now.

Simply removing (or changing) USE_SYSTEM_JEMALLOC=yes in debian/rules should be
enough.

No problem. :)


Regards,

I have also been bitten by this.

Since activedefrag cannot currently be enabled in the debain build, redis
will eventually consume all system memory (or reach its configured
maxmemory) and eventually crash.

activedefrag allows redis to cleanup its memory usage, without
activedefrag it will eventually just crash or cause an OOM event if the
internal maxmemory var config isnt set.

redis activedefrag notes:
https://cloud.google.com/memorystore/docs/redis/memory-management-best-practices#active_defragmentation

The issue mainly appears when redis keys are being refreshed often with
small quantities of data, for example counters being refreshed every few ms
within a month we have seen redis memory usage bloom to 15gb for a dataset
that when exported or saved is only a few hundred mb, the fragmentation is
quite large and is not being cleaned up.

Jamie