Problem with IMAP Server and Header Cache

Thu Jan 26 16:07:50 UTC 2023

Am 2023-01-26 00:33, schrieb Kevin J. McCarthy:

> Hmmm... I see in the log that even in the "working" case it's taking a
> good 15 seconds to load in the messages from the seqset/header cache.
> It's a big mailbox, though, so maybe that's to be expected.
> 
> I noticed you are using 1.13.2.  Are you in a position to try building
> a newer version of mutt?  There have been several important fixes to
> QRESYNC since then (including a bug in the seqset iterator).  It
> doesn't *look* like that's the issue here, but it would still be a
> good idea to try 2.2.9 if possible.
> 
> While you are at it, you might also try building with a different
> header cache backend.  Perhaps tokyocabinet is having issues.  I would
> suggest kyotocabinet or perhaps lmdb and see how they work.
> 
> On ubuntu, just run
>   $ sudo apt build-dep mutt
>   $ sudo apt install libkyotocabinet-dev liblmdb-dev
>   $ ./configure --enable-compressed --enable-debug --enable-fcntl \
>        --enable-hcache --enable-gpgme --enable-imap --enable-smtp \
>        --enable-pop --enable-sidebar --with-curses --with-gnutls \
>        --with-gss --with-idn --with-sasl \
>        --with-kyotocabinet      (or --with-lmdb)
>   $ make
>   $ sudo make install

I tried building building latest master with --with-kyotocabinet

It took me a bit until I found `./prepare` but then I could run 
configure
(and for some reason, it didn't like my installed `libidn2-dev` library,
so I build without `--with-idn`, but I suppose for the test it doesn't 
really
matter.)

#v+
Mutt 2.2.9+62 (90236f5a) (2022-12-20)
Copyright (C) 1996-2022 Michael R. Elkins and others.
Mutt comes with ABSOLUTELY NO WARRANTY; for details type `mutt -vv'.
Mutt is free software, and you are welcome to redistribute it
under certain conditions; type `mutt -vv' for details.

System: Linux 5.4.0-137-generic (x86_64)
ncurses: ncurses 6.2.20200212 (compiled with 6.2)
hcache backend: kyotocabinet 1.2.76

Compiler:
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/10/lto-wrapper
OFFLOAD_TARGET_NAMES=nvptx-none:amdgcn-amdhsa:hsa
OFFLOAD_TARGET_DEFAULT=1
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Debian 10.2.1-6' 
--with-bugurl=file:///usr/share/doc/gcc-10/README.Bugs 
--enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 
--prefix=/usr --with-gcc-major-version-only --program-suffix=-10 
--program-prefix=x86_64-linux-gnu- --enable-shared 
--enable-linker-build-id --libexecdir=/usr/lib 
--without-included-gettext --enable-threads=posix --libdir=/usr/lib 
--enable-nls --enable-bootstrap --enable-clocale=gnu 
--enable-libstdcxx-debug --enable-libstdcxx-time=yes 
--with-default-libstdcxx-abi=new --enable-gnu-unique-object 
--disable-vtable-verify --enable-plugin --enable-default-pie 
--with-system-zlib --enable-libphobos-checking=release 
--with-target-system-zlib=auto --enable-objc-gc=auto --enable-multiarch 
--disable-werror --with-arch-32=i686 --with-abi=m64 
--with-multilib-list=m32,m64,mx32 --enable-multilib --with-tune=generic 
--enable-offload-targets=nvptx-none=/build/gcc-10-Km9U7s/gcc-10-10.2.1/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-10-Km9U7s/gcc-10-10.2.1/debian/tmp-gcn/usr,hsa 
--without-cuda-driver --enable-checking=release --build=x86_64-linux-gnu 
--host=x86_64-linux-gnu --target=x86_64-linux-gnu 
--with-build-config=bootstrap-lto-lean --enable-link-mutex
Thread model: posix
Supported LTO compression algorithms: zlib zstd
gcc version 10.2.1 20210110 (Debian 10.2.1-6)

Configure options: '--enable-compressed' '--enable-debug' 
'--enable-fcntl' '--enable-hcache' '--enable-gpgme' '--enable-imap' 
'--enable-smtp' '--enable-pop' '--enable-sidebar' '--with-curses' 
'--with-gnutls' '--with-gss' '--with-sasl' 
'--prefix=/home/chrisbra/local' '--with-kyotocabinet'

Compilation CFLAGS: -Wall -pedantic -Wno-long-long -g -O2

#v-

I wiped out the header-cache, to make sure there are no 
incompatibilities, re-connected locally
build up the header-cache again (which takes a while to download all 
messages) browsed quickly
and quit mutt and did a fresh reconnection, so that the new header-cache 
will be used again.

But I am afraid, I am seeing the same problem (I haven't waited yet 3 
hours until mutt gets disconnected
from the IMAP server, but the symptoms look exactly the same.

Mutt just shows me "Selecting github..." and does "something" :/

In the debug file I see:

#v+
[2023-01-26 16:52:40] Mutt/2.2.9+62 (90236f5a) (2022-12-20) debugging at 
level 2
[2023-01-26 16:52:40] In mutt_reflow_windows
[2023-01-26 16:52:40] In mutt_reflow_windows
[2023-01-26 16:52:40] In mutt_reflow_windows
[...]
# Just so it's clear its the new mutt 2.2 :)
[...]
[2023-01-26 16:52:45] New mail in =vim, =zsh-users, =Trash, =spam
[2023-01-26 16:52:47] Mailbox is unchanged.
[2023-01-26 16:52:47] 4> DONE^M
a0042 CLOSE^M
[2023-01-26 16:52:47] 4< a0041 OK Idle completed (2.993 + 2.993 + 2.992 
secs).
[2023-01-26 16:52:47] 4< a0042 OK Close completed (0.001 + 0.000 secs).
[2023-01-26 16:52:48] Reading configuration file 
'/home/chrisbra/.mutt/profile.256bit.de'.
[2023-01-26 16:52:48] Reading imap://localhost/github...
[2023-01-26 16:52:48] Selecting github...
[2023-01-26 16:52:48] 4> a0043 STATUS "Entwurf" (MESSAGES)^M
a0044 SELECT "github" (CONDSTORE)^M
[2023-01-26 16:52:48] 4< * STATUS Entwurf (MESSAGES 0)
[2023-01-26 16:52:48] 4< a0043 OK Status completed (0.001 + 0.000 secs).
[2023-01-26 16:52:48] 4< * FLAGS (\Answered \Flagged \Deleted \Seen 
\Draft)
[2023-01-26 16:52:48] 4< * OK [PERMANENTFLAGS (\Answered \Flagged 
\Deleted \Seen \Draft \*)] Flags permitted.
[2023-01-26 16:52:48] 4< * 113867 EXISTS
[2023-01-26 16:52:48] Handling EXISTS
[2023-01-26 16:52:48] cmd_handle_untagged: New mail in github - 113867 
messages total.
[2023-01-26 16:52:48] 4< * 0 RECENT
[2023-01-26 16:52:48] 4< * OK [UNSEEN 4] First unseen.
[2023-01-26 16:52:48] 4< * OK [UIDVALIDITY 1384369187] UIDs valid
[2023-01-26 16:52:48] 4< * OK [UIDNEXT 351346] Predicted next UID
[2023-01-26 16:52:48] 4< * OK [HIGHESTMODSEQ 166848] Highest
[2023-01-26 16:52:48] 4< a0044 OK [READ-WRITE] Select completed (0.009 + 
0.000 + 0.008 secs).
[2023-01-26 16:52:48] Reading uid seqset from header cache
#v-

It's now only 10 minutes without progress, but I don't expect any 
changes/progress within the next 3 hours :)

Thanks for having a look Kevin,

Chris, now using roundcube to send the mail, since mutt is, well busy