S-nail announcement

S-nail v14.9.24 (“Black ships ate the sky”)

Hello list,

hereby i announce S-nail v14.9.24 "Black ships ate the sky".
It is a bugfix release (Microsoft IMAP servers should now work).

Credits, in order of commit appearance: Paul Eggert, Madou Mad,
Paride Legovini, Jesse Alama.

We welcome Madou Mad and Jesse Alama in THANKS.

Number games

The release commit is [v14.9.24.ar] on [release/v14.9.24],
and it has also been stored on the [timeline].
The [master/stable/v14.9] branch was tagged [v14.9.24].

The git(1) release commits and tags, as well as the release balls
have been signed with the OpenPGP signing subkey EEC8C2FF of key

  steffen@sdaoden.eu  /  1883A0DD
  (EE19 E1C1 F2F7 054F 8D39  54D8 3089 64B5 1883 A0DD)

Release balls and OpenPGP signatures (.asc) are accessible via
https?://ftp.sdaoden.eu, also the key steffen.asc (+ GnuPG WKD).
Inline copies of the signatures are at the end of this message.

    SHA1 11bafd0dbb78b45f36f802693233499a9f307e37
  SHA256 2714d6b8fb2af3b363fc7c79b76d058753716345d1b6ebcd8870ecd0e4f7ef8c
  SHA512 03f6a6f446391b6f91ed3c8875c3e7fdfac9d4e77ea1d52a7e98aa84cfd0edae137d5b9afba3bdc9a31ab67cee5237930b74b42ae3acb54aee4758553a4f1df2
    SHA1 1f4f607fde829356e9a7dbf6e9638b313b669bb0
  SHA256 4e4db869792bcdcc97fae1db1a6641e664a93b2e85f6eac53e7b3ac305e6c586
  SHA512 15101019a9c14fef8162fbce1a15133d2c70f27476b08a7f183d195fb52ed2854731984f833236c543fd681e299c3d93ec508154573b5897d707a0ec5c216575

These files and this announcement text are available as "-latest"
symbolic links, for example s-nail-latest.txt.

  Announcement : https?://www.sdaoden.eu/code-nail-ann.html
  Manual       : https?://www.sdaoden.eu/code-nail.html
  Web          : https?://www.sdaoden.eu/code.html#s-mailx
  git(1) clone : https?://git.sdaoden.eu/scm/s-nail.git
  git(1) browse: https?://git.sdaoden.eu/browse/s-nail.git

S - n a i l / S - m a i l x  N e w s

  mdocmx(7) anchors are denoted by a number-sign #: typing
  "^A ANCHOR" while reading the man(1)ual in a capable less(1)
  will scroll to the manual's Point-Of-Interest, and pointing
  a web browser to the "#ANCHOR" of the online manual works.

NOTES, ChangeLog (packager-affine)

- Release tarball might be in --format=pax.  (Paul Eggert)

- Again honour MTA arguments passed after the "--" command line
  "option" in non-send-only mode (broken since at least v14.9.12).

- Thanks to Madou Mad we should now support Microsoft IMAP
  servers.  The codebase treated untagged and tagged responses
  alike: this never worked!  (Madou Mad)
  (P.S.: no credits to Steven or Gavin here.)

- Fix crash due to crafted emails.  (Our RFC 2047 MIME handling
  is still terrible, but we should not crash no more.)


- Fix *expandargv*=restrict (broken since v14.9.0).

- Fix a bug from nail 10.08 as of 2004-06-28 that leads to
  crashes when some file cannot be loaded.
  And while here fix one file-close overshoot of mine in since
  v14.9.16, that would cause panics in debug-enabled code.

- Like anyone else we now deal with argc==0 aka CVE-2021-4034.

git(1) shortlog (edited)

Steffen Nurpmeso (28):
096665105b mk/make-release.inc: try to use tar(1) --format=pax (Paul Eggert)
b2bfa82a00 Fix expandargv=restrict (since 2017-01-15!)..
7ea98f69b7 FIX MTA args after -- for non-send-only mode!..
b6736900e8 mx-test.sh: change UTF-8 locale order AGAIN!
294ed7f97c imap_response_parse(): stop overbalancing untagged responses (Madou
c4b7512fb6 THANKS: Madou Mad
9352ebfe6e FIX/tweak previous, add a_imap_res__untagged() (Madou Mad)
561e078c1c Reproducibility detection a_main_setup_vars() -> a_main_startup()..
14c4dbf89c mx_fs_open_any(): FIX !fs_file_load() crash (since nail 10.08,
13e06c8fe1 mx_smime_split(): fix *FILE** "return" upon error return..
3dd2f43372 main(): work nicely with argc==0 (after reading CVE-2021-4034)
0386c021d3 ps-dotlock: call setgid(), too (actually Paride Legovini)
987f73fad5 a_termios_sig_adjust(): fix "unshare -f mailx" ^C<RET>^C crash
f336d267fb mime.types: move .xsl from xml to xslt+xml (Jesse Alama)
b45c897315 THANKS: Jesse Alama
d53396df1e mime_fromhdr(): FIX crash in crafted mail (since ~2013)..
aca800c3af mx_mime_display_from_header(): FIX previous
7a7e48cb36 n_iconv_open(): FIX: name normalization can fail!

v14.9.23 ("Tits look ahead for winter"), 2021-11-11

Credits, in order of commit appearance: Noctambule, Stephen Isard,
Andreas Teuber, Simon Gerraty, Geoff Clare, Robert Elz,
Harald van Dijk, Bryan Drewery, Andrea Biardi, Jörg Schilling.

We welcome Andreas Teuber, Harald van Dijk, Bryan Drewery and
Andrea Biardi in THANKS.

Good-bye and farewall dear Jörg!

NOTES, ChangeLog (packager-affine)

- VAL_RANDOM=getentropy was broken.

- An unset *mta-bcc-ok* could cause partial Bcc: content
  to be written to Cc: or To:.  (If so many addressees where given
  that multiple lines have to be used, content of all lines but
  the first would be written.)  (Andreas Teuber)

- Date offset calculation was wrong for timezones which use
  a negative adjustment for "is daylight saving time active".
  (Andrea Biardi)


- Continue searching MIME handler sources if the first found one
  is not applicable in current context.  (Noctambule)

- Fixed `vexpr' shift operators broken for "optimization" :-(.
  Also fix `date-utc' subcommand which used two "dutc_month"
  instead of one plus "dutc_day".

- Temporarily set an unset $LESS to portable "RI" not "RXi".

- A couple of SIGALARM / alarm(2) fixes for the -keepalive-
  variable series.  (Stephen Isard)

- We drop inherited effective IDs upon startup.  (Harald van Dijk)

- `mimeview' now also asks for text/ parts whether an action
  shall be applied.  And if there is no plain part we may
  use the rich one if possible.  (Stephen Isard)

- *pipe-TYPE/SUBTYPE* now support type-markers.  (Noctambule)

git(1) shortlog: Steffen Nurpmeso (57)

v14.9.22 ("Feathery sound of close tit flypast"), 2021-02-24

O tempora!  O mores!
Three fixes for thoughtless and also in other ways superficial
code changes i have done.
Apologises to all, but especially to Mr. Bell.

Credits, in order of commit appearance: Jens Schleusener,
Olav Mørkrid, Russell Bell and Johannes Schöpfer.

A special thanks to Olav Mørkrid is overdue.


- After "echo foo | s-nail $USER" we no longer leave the terminal
  in a different state than it should be.  Terminal initialization
  was not rethought from scratch after we focused on the MLE
  editor only and after we have implemented our own termcap/info
  layer .. now done, and -A account switch and -X commands now run
  properly covered already, too.  (Olav Mørkrid)

- I had implemented a thoughtless use of close_range() aka
  closefrom() -- this was totally broken (mysteriously not covered
  by the tests, have to look, will fix for the future, too).  We
  can never do this, and if it is only for `readctl'.
  (Russell Bell, Johannes Schöpfer)

git(1) shortlog: Steffen Nurpmeso (8)

v14.9.21 ("Tit escapes with a peanut"), 2021-01-21

I apologise for the inconvenience of yet another bugfix release.
It fixes a possible SMTP buffer overflow triggerable by
a malicious server as reported by Olav Mørkrid, and a socket code
memory access error that can be seen when using IMAP on at least
Solaris, which was reported by Jörg Schilling.

Yasuhiro Kimura helped to unconfuse problems that i attributed to
BSD make, but which were indeed shell errors, most notably mksh.
Excuses are due to the forgiving and friendly Simon Gerraty (BSD
make) and Paul Smith (GNU make).
(In the end test job reaper code has been rewritten completely.)

Credits, in order of commit appearance: Simon Gerraty,
Yasuhiro Kimura, Matthias Gerstner, Olav Mørkrid, Jörg Schilling,
and Geoff Clare.

We welcome Simon Gerraty and Matthias Gerstner in THANKS.

We are  https://scan.coverity.com/projects/s-nail (project 444).

NOTES, ChangeLog (packager-affine)

- The test now can skip individual tests included in $SKIPTEST,
  for example "# make test SKIPTEST='eval vexpr'.

- New option VAL_ERRORS_LIMIT (by default enbaled), and new
  variable *errors-limit* to configure error ring size.

- We become even more portable to SysV/Solaris.  (Jörg Schilling)


- Without termcap/terminfo support the outermost column was always
  made accessible (since "not not" defining "am" in *termcap* is
  not possible, bug since v14.9.12), on the other hand the "ch"
  builtin implementation never reached out to this last column
  (since ever).

- Manual: after rewrite and review i think the manual section
  "Character sets" is of acceptable quality.  (Jörg Schilling)

git(1) shortlog: Steffen Nurpmeso (56)

v14.9.20 ("Sombre Tit (Trauermeise)"), 2020-12-12

Fixes things like TLS over SOCKS and too many other bugs.
We have some new things, too.

Credits, in order of commit appearance: Roberto Ricci,
Aharon Robbins, Kevin McCarthy, Predrag Punosevac,
Paride Legovini, Olav Mørkrid, Ron Varburg, Jürgen Daubert,
Russell Bell, and Geoff Clare.

It is courtesy to give a special credit to Coverity.com, even
though it only found false positives.
( https://scan.coverity.com/projects/s-nail, project 444.)

We welcome Roberto Ricci, Ron Varburg and Geoff Clare in THANKS.

NOTES, ChangeLog (packager-affine)

+ FreeBSD, DragonFlyBSD, (NetBSD): there are bugs in BSD make and
  FreeBSD-derived sh(1)ells regarding sh(1) monitor mode that is
  now used in mx-test.sh ("make test").
  Please pass in JOBMON=n on make(1) command line, or use
  a different shell (and make).

  (You could also pass in a different JOBWAIT= now.)

+ Binaries are now installed 0755 not 0555.  (Jürgen Daubert)

+ On SunOS/Solaris we now use the normal $CC detection algorithm.
  (No longer forcefully overwrite, prefer open source compilers.)

- TLS connections can now be proxied via *socks-proxy*.

- `echo' no longer performs "Filename transformations".
  Introduced in BSD Mail in 1988, but bad.
  Just use `vexpr' file-expand.

- `local' now works like `localopts' when used
  with `set' and `unset'.
  This is true for built-in variables only, of course, using
  `local' for those was forbidden in the past, i think.
  Note that `localopts' will be obsoleted in v14.10, we will only
  use modifiers in the future.


- EXTERNAL authentication over IMAP and POP3 was fixed.

- *tls-fingerprint* is now tested case-insensitively.

- *quote* gained an "allbodies" keyword.

- "The Mailcap files" gained "x-mailx-last-resort" and
  "x-mailx-ignore" flags.  (Latter: Russell Bell)

- `history' "delete" can now delete multiple entries per

git(1) shortlog: Steffen Nurpmeso (110)

v14.9.19 ("Tufted titmouse (Indianermeise)"), 2020-04-26

Fixing an unknown-8bit/iconv(1/3) misbehaviour when displaying
mails, and making this MUA ready for OpenSSL 3.0.

Credits, in order of commit appearance: Anirudh Oppiliappan and
Claus Assmann.

We welcome Anirudh Oppiliappan and Claus Assmann in THANKS.

git(1) shortlog: Steffen Nurpmeso (14)

v14.9.18 ("It is spring time, what a bliss"), 2020-04-18

Some bugfixes and tweaks that accumulated over the months.

Credits, in order of commit appearance: Jens Schleusener,
Noctambule, Kyle Evans.

It is courtesy to give a special credit to Coverity.com.
(< https://scan.coverity.com/projects/s-nail>, project 444.)

We welcome Noctambule and Kyle Evans in THANKS.

NOTES, ChangeLog (packager-affine)

- Warning: we _will_ have v15-compat=yes as a default in v14.10!

- Fix for `~f', `~m', `~Q', `~U' and `~u':
  no longer include all MIME parts (bug since at least v14.9.16).

- Fix for handling of "The Mailcap files":
  a single copiousoutput/x-mailx-tmpfile-fill|nametemplate
  combination was falsely handled, mostly affecting binary file
  formats (which got broken by applied character set conversion).

- Fix for automatic S/MIME encrypted key / certificate password
  lookup: used $LOGNAME@[no hostname].smime-cert-key as
  a fallback, instead of using the value of *from*.

- `~R' and `~r', as well as all prompts which expect
  file names and did not yet do so, expect shell-quoted names.
  (See "Shell-style argument quoting".)


- `~Q' now acts as if *quote* is set.
  Does not really make sense otherwise.

- New environment variable $SOCKS5_PROXY is tight together
  with *socks-proxy*: setting the one affects the other.
  This $SOCKS5_PROXY is introduced by FreeBSD to affect all
  programs which are capable to proxy via SOCKS5 (it seems).
  (Kyle Evans)

git(1) shortlog: Steffen Nurpmeso (26)

v14.9.17 ("To bind, or not to bind.."), 2020-02-02

A shadowed key bindings report on bash-bugs@ made me aware that
our `bind' code has never seen a real review, and not only were we
incapable too, but the code was a piece of shit, when i looked at
it.  So please find here a bugfix release to smoothly end v14.9.

Credits, in order of commit appearance: Jens Schleusener,
Johannes Schöpfer, Russell Bell, and Koichi Murase.

It is courtesy to give a special credit to Coverity.com.
(< https://scan.coverity.com/projects/s-nail>, project 444.)

We welcome Koichi Murase in THANKS.

And now for something completely different.

NOTES, ChangeLog (packager-affine)

- "make test" now works when run by root or on read-only
  file-systems.  (Johannes Schöpfer)

- Our `bind' now supports overall key-sequence timeouts,
  as opposed to inter-byte timeouts.  For this i have obsoleted
  *bind-timeout* in favour of *bind-inter-byte-timeout*
  and *bind-inter-key-timeout* (not set by default).

  The bind tree code had two bugs, for one the "shortcut"
  key-bindings would all have been created in the "base" context,
  not where they really belong ("default" and "compose").
  And due to false list relinking shadowed key bindings did not
  work.  (Koichi Murase)

  When used with 3x*verbose*#? (aka -v) the bind tree
  is now dumped when it has been build (once used first, and after
  modifications.  Putting all this together, a resource file

    cat >/tmp/t.sh <<'__EOT'
    cat >/tmp/t.rc <<'_EOT'
    set line-editor-no-defaults
    bind base $'\n' mle-commit
    bind base $'\c?' mle-del-bwd
    bind base $'\cT' echo one
    bind base $'\cT',$'\cT' echo two
    bind base abc echo 0
    bind base ab,c echo 1
    bind base abc,d echo 2
    bind base ac,d echo 3
    bind base a,b,c echo 4
    bind base a,b,c,d echo 5
    bind base a,b,cc,d echo 6
    set quiet noheader
    set bind-inter-key-timeout=2500
    set bind-inter-byte-timeout=250
    bind base a,b,c,d # now works, too!
    MAILRC=/tmp/t.rc $MAILX -R:u -Y 'set verbose=3'
    rm -f /tmp/t.sh /tmp/t.rc
    sh /tmp/t.sh

  will now do the expected.

- `Reply': no longer honours *recipients-in-cc*


- Manual: after review i think the following sections are of
  acceptable quality: "On terminal control and line editor",
  "Coloured display".

- *verbose* is no longer a boolean but can be assigned
  a numeric value.  But "set verbose verbose verbose"
  also still works.

- New variable *reply-to-swap-in* tries to work around
  when `reply'ing (or `Reply'ing) to such a message.
  It tries to move the HUMAN into responsibility.

git(1) shortlog: Steffen Nurpmeso (65)

v14.9.16 ("Message of Winter, your hopes shall be crushed"), 2019-12-29

At the end of the v14.9 series we finally deliver the RFC 1524
mailcap support, as well as other improvements, and new features.
Many bugfixes and tests arrive.
A whole bunch of things are backward-incompatible, but i would
assume that most use cases are not at all affected.

Credits, in order of commit appearance: Brian Evans,
Paride Legovini, Yasuhiro KIMURA, John P. Linderman, Leo,
Martin Neitzel, Alexander Harm, Ken Hornstein, Martynas Bendorius,
Russell Bell, Goesta Smekal, Kevin McCarthy, Ralph Keller,
Ralph Corderoy, Viktor Szépe, Jelle van der Waa, Arnout Engelen,
Stuart Henderson, elo, and Benjamin A. Wong.

We welcome Yasuhiro KIMURA, John P. Linderman, Leo,
Martynas Bendorius, Goesta Smekal, Jelle van der Waa,
Arnout Engelen, elo, and Benjamin A. Wong in THANKS.

NOTES, ChangeLog (packager-affine)

- Renamed *expandaddr* namehostex to nametoaddr.  (Better.)

- *tls-features* string now starts and ends with comma.
  (Just like *features* does.)

- -L longname --header-search renamed to --search.

- Config option OPT_FORCED_STACKPROT=xy now
  OPT_AUTOCC_STACKPROT=y, and now auto-enabled for OPT_AUTOCC=y
  (even though i hate that, protectors in shipout code..).

- `account' return value now matters, and can thus be used to
  abort account switching.

- -: has new "x" mode, which executes the directives of the
  compiled-in resource file.  (The template content is now
  compiled in too, so the real file does not need to be loaded.)
  (Brian Evans)

- Quote etc. injections (*quote-inject-head*) now always
  happen (if set), regardless of *quote*.

- We now truly honour POSIX command abbreviations (order).
  As part of that i finally implemented a simple command lookup
  speedup, these are now almost alphabetical (unless abbreviations
  prevent it).

- Added *mta-bcc-ok*.  Bummer.  It seems exim and courier
  do not remove Bcc: headers as required by standards unless
  invoked with a special command line argument.  So we now
  do not pass Bcc: headers to file-based MTAs unless this
  variable is set explicitly.  (Kevin McCarthy).

- "COMMAND ESCAPES" now have a $ command modifier, which
  causes a shell-style `eval'uation before the command
  escape is executed.

  And so `~<', `~R' and `~r' no longer expand $VAR
  expressions by themselves.

  And so -a, `~@', and "attachment insert" of `~^'
  and `digmsg' only perform ~/-style expansions.

- `~^': use shell-style argument expansion.
  We handled `~^' and `digmsg' differently, but furthermore
  turned the shell-parsed data of the latter into whitespace
  separated data, which made it impossible to, for example,
  use attachments with whitespace in their names.

  This increases the interaction protocol version number of
  *on-compose-splice* from "0 0 1" to "0 0 2"!
  Because, we do also quote the output, since using `read'
  (or read(1)) causes *ifs* ($IFS) normalization.
  All that could be done would be (for ourselves):

      define x {
        set ifs=
        read vany
        unset ifs
        vpospar set $vany
        # And now assign the desired real fields
        set real-var1=$1 real-var2=$2  ...

  Sick!  So instead introduce a `readsh' command which works
  like `read' but splits fields at shell token boundaries,
  for example from within *on-compose-splice*:

    echo '~^h s subject'; read stat name; readsh sub; read i

  There would be better examples.  (Ralph Keller)

- `~F', `~f', `~M', `~m', `~U' and `~u' now
  honour *forward-inject-head* and *forward-inject-tail*.

- New option OPT_MAILCAP, by default enabled.
  Disable at runtime via *mailcap-disable*, all documented
  in "The Mailcap files".

- OpenBSD: really auto-find number of processors in test script.
  Work around fflush(3) not adapting POSIX behaviour.

- Manual: after review i think the following sections are of
  acceptable quality: "Encrypted network communication",
  "A starter", "On URL syntax and credential lookup",
  "The Mailcap files", and
  "But, how about XOAUTH2 / OAUTHBEARER?".

  The latter is actually (Stuart Henderson, Benjamin A. Wong),
  and now provides a copy+paste example of how to keep a
  OAUTHBEARER token up-to-date with S-nail (with some care).
  Yes, it is terrible.

- Default .rc file: keeps Sender: by default (Ken Hornstein),
  sets *followup-to-honour* and *reply-to-honour*,
  and gives more *history-gabby*ness.


- Add `Lfollowup'.  (Russell Bell)

- `?': prefix \ (quoted!) to command to avoid
  `commandalias'  matching:

  ? ? S
  S -> spamspam: Teach the spam detector that <msglist> is spam
  ? ? \\S
  S (Save): Like `save', but derive filename from first sender

- `history' has new "delete NUMBER" subcommand.

- Add *forward-add-cc* and *quote-add-cc* to Cc: the
  originator of a forwarded or quoted message, respectively.
  (Goesta Smekal)

- *history-gabby* now has a value to allow for more.
  This changes second argument of *on-history-addition*
  from boolean to context string.

- New command `mtaaliases'.  We no longer automatically
  update the *mta-aliases* cache.  (Maybe much later we will
  have a path_monitor or something, until then, not.)

  And `netrc' "load" is now indeed "clear" + "load".

  `netrc' also gained a "lookup" subcommand.  (Ralph Corderoy)

- *headline* %L format will announce possibility that
  a message could be a list.

- `chdir', `rename' and `remove' use shell-style
  argument quoting.

- `tls' gained "certchain" and "certificate" subcommands.

- `folder' can open RFC 5322 messages via eml:// protocol,
  as in "folder eml:///tmp/msg.eml".  Yet primitive and only
  read-only.  (Viktor Szépe)

- `vexpr' gained "date-utc", "date-stamp-utc" and "epoch"
  subcommands.  (Benjamin A. Wong)

- *on-main-loop-tick* now also happens for commands passed
  via -Y.

git(1) shortlog: Yasuhiro KIMURA (1), Steffen Nurpmeso (263)

v14.9.15 ("Tit family in the trees"), 2019-08-18

Plugging a bug regarding copying data out of invalid MBOX mail
databases which is present in all BSD Mails and in Unix V10 mail,
and bringing in some tweaks, this update hopefully really marks
the end of the v14.9.* series.

After more than four and a half years i again have a VM testbed,
with an increasing number of VM combinations.  (Yet still too few,
but nonetheless, a dramatical improvement.)  This includes
a GSS-API testbed, with an ArchLinux server and Linux and FreeBSD
clients (do not ask why no additional FreeBSD server, i want to
use binary packages).  This brought GSSAPI tweaks.

Credits, in order of commit appearance: Ralph Corderoy,
Chet Ramey, Robert Elz, Jilles Tjoelker, Steve Izma, Viktor Szépe,
and Jean-Marc Pigeon.

Very special thanks go to Tarqi Kazan and Ivan Vučica, who tested
GSS-API in the past until it worked (again), testing against my
blind flight patches!  Thank you very much, guys!

We welcome Chet Ramey, Jilles Tjoelker and Steve Izma in THANKS.

NOTES, ChangeLog (packager-affine)

- *features* and *tls-features* are now prefixed with
  a comma ",", not with the number sign "#" (which could
  increasingly `eval'uate to a comment).

- We now support parallelized tests.  It takes a numeric job
  number out of $MAKEFLAGS, or tries to fetch the number of
  processors otherwise (really!).  To go singleprocessor
  "$ make testnj" has to be called explicitly.
  With or without, we will terminate tests which take too long.

  This is truly tremendous, on the unstable9s machine of the
  OpenCSW.org cluster for example we now need 24 seconds instead
  of by far more than 300.  What a release!

  With help of (Chet Ramey, Robert Elz, Jilles Tjoelker and
  Steve Izma)

- EXTERNAL authentication is truly a mess.  It has been fixed for
  POP3, where it was broken on our side.  But it seems the
  internet does not like that, or cannot (pass user credentials
  from a certificate gracefully to the authenticator).
  Anyway.  I have introduced EXTERNANON in addition for all of
  IMAP, POP3 and SMTP.  This could now result in a usable
  combination, regardless of what server(s) are contacted.

- The hook *on-account-cleanup* will now be called even
  upon program exit (i.e., implicitly leaving the account).


- New *followup-to-add-cc* will place the user in the Cc:
  list if it will place her in the Mail-Followup-To:.

- New hook *on-program-exit*.

- *pop3-auth*=gssapi is now supported.
  For IMAP, SASL-IR will be used for GSSAPI if possible (saving
  a packet round-trip).

- *expandaddr* has the new keyword "namehostex".
  If set, plain name addressees, like "To: steffen", will be
  expanded to NAME@HOSTNAME (where the latter could be
  *hostname*) if NAME is a valid user on the current host.
  (Viktor Szépe, Jean-Marc Pigeon)

git(1) shortlog: Steffen Nurpmeso (45)

v14.9.14 ("Great tit passed moult"), 2019-07-27

This is an unwanted and unplanned but unfortunately necessary
bugfix release.  I hope it marks the end of the v14.9.* series.

I presume you would be surprised if it would not also bring some
features, this time mostly support of MTA-style aliases as
inquired by Jean-Marc Pigeon, some authentication work (XOAUTH2/
OAUTHBEARER support), and as usual development to the last minute.

It fixes IMAP GSSAPI authentication, thanks to Ivan Vučica for
reporting and testing this issue (Debian #930691; still have no
testbed, but will soon!), and imap-delim, which i broke in July
2017, thanks to Ralph Keller for repetitive reporting.

For OpenBSD and SunOS 5.9 this release fixes long standing (must
be) race conditions regarding child processes and their I/O setup.

  Never seen before, but my new box (i stepped a decade of
  hardware improvements, finally) rather regulary has shown them
  when running the test suite.  (On the OpenCSW cluster my speed
  varies, but i had a very good day and seen them there once.)

  This (finally) caused the complete rewrite of the child process
  (and termios) handling that i (had to) mention in communication
  with Gavin Troy already back in, i do not know -- 2013?  (Still
  not event loop based, but near getting good feelings there.)

  Funnily the problem (child descriptors were closed by the parent
  before the fork(2)ed childs had the opportunity to dup(2)licate
  their file descriptors) reminded me of a message of the german
  computer magazine c't, maybe around 2001/2002, when OpenBSD
  improved their fork(2) performance in a day or two after having
  appeared declassified in a comparison with other OSes.
  (Of course it was nothing but our own fault to not synchronize on
  the child, but blindly assuming that a fork(2) child gets the
  opportunity to run immediately.)

  Dear Predrag: would it now be possible for you to upgrade from
  v14.8.12?  I really would like to know!

Credits, in order of commit appearance: Martin Lucina,
Viktor Szépe, Alexander Harm, Anders Magnusson, Thomas Haigh,
Martin T, Ivan Vučica, Nicholas Marriott, Alexander Harm,
Steven Penny, Jean-Marc Pigeon, Martin Neitzel, Paul Vojta,
Russell Bell, Paride Legovini and Ralph Keller.

A special credit to Coverity.com once again, it found bugs!
(< https://scan.coverity.com/projects/s-nail>, project number 444.)

We welcome Martin Lucina, Anders Magnusson, Thomas Haigh,
Martin T, Ivan Vučica, Nicholas Marriott, Steven Penny and
Ralph Keller in THANKS.

NOTES, ChangeLog (packager-affine)

- The (very backward) Debian mawk is now supported directly.
  (Martin Lucina, Viktor Szépe)

- GCC (8.3.0) -Os inlining bug (wmt) is worked around.
  And GNU awk 5 warnings have been fixed (before CRUX).

- "|PIPE RECEIVER" errors seen on SunOS 5.9 and OpenBSD have first
  been fixed, and then caused a major rewrite of the child and
  termios handling for a rather "real" fix.

  The latter has the side effect that *pipe-TYPE/SUBTYPE*
  handlers will now have their standard output go to /dev/null.

- IMAP GSSAPI authentication should work again.  (Ivan Vučica)

- -C testable can be used more than two times.

- The "grappa" mode of mk/make-release.sh can now be used
  "everywhere".  See INSTALL on interest.  (It now gracefully
  fails if s-nail is not installed: we need that for hashing.)

- New option OPT_MTA, by default enabled.
  Set *mta-aliases* to a valid path in aliases(5) syntax,
  and we will expand them.  All Postfix directives but :include:
  are supported.  Only clear text files are supported, no DBs.
  (Jean-Marc Pigeon)

- `~^' will now verify *expandaddr* right away in
  "~^ head ins to|cc|bcc", as is already done for `~t',
  `~c' and `~b'.

- OPT_AGENT and OPT_SPAM_SPAMD are gone; they were obsoleted on
  2017-07-16, and it is not expected to become noticed.

- *sendwait* is now initially set, and it gained an optional
  value, a comma-separated list of case-insensitive strings naming
  specific subsystems for which synchronousness shall be ensured
  (only).  Possible values are "mta" for *mta* delivery,
  and "pcc" for command-pipe receivers.

  P.S.: you can get a list of all initial values plus via

    $ s-nail -:/ -v -Xset -Xx

- Colours may now happen even in quickrun mode (-e,
  -H, -L)!  I thought it is ok nonetheless, because
  we i think always documented to enwrap `colour' setting
  in an according `if', as in

    \if terminal && [ "$features" =% +colour ]
      \colo iso view-header fg=red

- OPT_SOCKETS has been renamed to OPT_NET.

- -Y is now well defined under all conditions, with tests:

    The commands will be evaluated successively in the given
    order, and as if given on the program's standard input --
    before interactive prompting begins in interactive mode,
    after standard input has been consumed otherwise.

- XOAUTH2 / OAUTHBEARER (OAuth 2.0 bearer token, RFC 6750)
  autentication is now supported for all protocols.
  New FAQ entry "But, how about XOAUTH2 / OAUTHBEARER?"
  For driving the necessary external update tool a new
  *on-main-loop-tick* hook has been introduced.
  (I am thinking about adding support for an optional built-in
  token refresh.)

- Support for the EXTERNAL authentication method has been
  introduced.  This is UNTESTED, though.  (I am still in the
  process of re-setting up my VM test environment.)
  We do not verify presence of a client certificate etc., but only
  that a TLS secured channel is active, when using this method.
  (We now *verbose* log the used TLS version and cipher, too.)

- *imap-delim* works again as advertised!  This was broken in
  [1b9897a9] ((BWDIC!) Fix *imap-delim* behaviour.., 2017-07-01),
  and i think i was pretty much irritated by then.  Sorry!!
  While here, take *imap-delim* into account for `imapcodec'.
  (Ralph Keller).


- Manual section "HISTORY" improved a bit.  (Thomas Haigh)

- New variable *line-editor-cpl-word-breaks* (yet a bit

- MLE: add mle-raise-{int,quit,tstp} functions.  Ie., raising
  those signals via ^C and ^Z is no longer hard-wired (in the
  MLE), but can be reassigned.  (Nicholas Marriott)

- The makefiles no longer contain any awk code, that all has been
  separated into files under mk/.  (Alexander Harm)

- We now have "test" *mta*, which dumps to standard output
  or optionally to a file, and honours *mbox-fcc-and-pcc*:

     $ echo text | s-nail -:/ -Smta=test -s ubject user@exam.ple
     $ </dev/null s-nail -:/ -Smta=test://./xy -s ub user@exam.ple

- -# now acts as if variables were set via -S, rather than
  as via `set', meaning that they are "frozen" for a while.

- We have a new `colour' mapping, "mle-error".
  It applies to error note in *prompt*, as well as to any
  other error logged to the terminal.  The latter is temporary
  until we gain a more fine grained logging facility, which then--
  likely--introduces an entire mapping family.

  Btw., for messages not explicitly generated (by, eg.,
  `echoerr'), we now also avoid writing adjacent duplicates,
  but rather do a syslog-like thing.  That error generator has
  been rewritten a bit, now more expensive, but looks right.
  (Martin Neitzel, Paul Vojta)

- The value of $COLUMNS seen in child processes is now the
  real one, rather than the one we use (not necessarily identical,
  dependent on termcap/terminfo/terminal support).

- There are now three -v / *verbose* levels, not two.

- The HTML filter knows more HTML 4.0 entities.  (Russell Bell)

git(1) shortlog: Steffen Nurpmeso (161)

v14.9.13 ("Blue tit's spiral marriage swoop"), 2019-03-08

Not so much happened as i was busy with other things in the second
half of 2018, and could not find a grip: at the source level some
efforts to turn this to mailx have taken place, including early
work on a code abstraction that will be shared with my (g)roff
clone: it was tiring to come back to old ideas that i have already
implemented multiple times and in different languages.  To mention
it for the curious.

So: this is a bugfix release, in fact it fixes a tremendous amount
thereof, [master] was ahead by 62 commits from [v14.9.11] (not all
bugs though, hrmhrm), but also with, well, a few new features, and
of course, development to the last minute. ^_^

  v14.9.13 replaces v14.9.12 from yesterday which would pick up an
  ISO C 2011 statement that is impracticable, thanks to Johannes
  Schöpfer and Jürgen Daubert for reporting this (i should test
  without OPT_AUTOCC more often.)  I have also fixed -T to match
  NEWS (manual and code did the opposite).
  v14.9.12 balls will be removed from the server.

Credits, in order of commit appearance: Jörg Schilling,
Paride Legovini, Olav Mørkrid, Ralph Corderoy, Rich Felker,
Predrag Punosevac, Russell Bell, Dirk-Wilhelm Peters,
Jean-Marc Pigeon, Warren Toomey, Cág, Martin Neitzel,
Dr. Werner Fink, Dr. Matthias St. Pierre, Kurt Roeckx, Mike Sharov,
Joan and Johannes Schöpfer.

Very special thanks go to Jean-Marc Pigeon of OSUKISS Linux, who
provided me access to a VZGOT container on the most "beefy"
machine i have ever had access to!  It is ever so astonishing to
have work done in 90 seconds on this supercomputer which requires
an hour or more here.  Thank you, Jean-Marc.

And thanks to Kimura-san not only my web server became accessible
via TLSv1.3 (out-of-line), but we also have landed on FreeBSD!
Kimura-san is the one who installed a sympathic cron job that
reminds me of cherry petals wafting in a warm spring breeze --
in winter!

We welcome Russell Bell, Jean-Marc Pigeon, Warren Toomey,
Dr. Matthias St. Pierre, Kurt Roeckx, Mike Sharov, Joan and
Johannes Schöpfer in THANKS.

NOTES, ChangeLog (packager-affine)

- Packagers could follow stable/* via git(1) and the "grappa" mode
  of the release script (see README); perl(1) is required to be
  totally en par with an official release.  (Paride Legovini)

    $ git checkout stable/stable
    $ sh mk/make-release.sh grappa mybranch
    Program version is [.], packager release addition shall be: xy
    Is s-nail <v[.]-xy> correct? [y/n] y
    Switched to branch 'mybranch'
    $ git commit -S -n -m 'My release [.]-xy'

- $MAKEJOBS vanished, just use -j or whatever your make(1) supports.
  Luckily the tested make(1)s can be persuaded to dig each others
  .WAIT / .WAIT: / .NOTPARALLEL: targets.. and do the right thing.

- $OBJDIR support added, i use it for building / testing on tmpfs.
  It works in conjunction with make-emerge.sh, too, thus
  out-of-tree out-of-tree is possible (more or less; see INSTALL).
    $ make tangerine OBJDIR=/tmp/x/y/z

- Option VAL_PRIVSEP_USER has been renamed to VAL_PS_DOTLOCK_USER
  (to reflect the new "deep tree" directory layout).

- Option OPT_QUOTE_FOLD has been renamed to OPT_FILTER_QUOTE_FOLD
  (to reflect later code changes upwards compatibly).

- New options OPT_CMD_VEXPR and OPT_CMD_CSOP, by default enabled.
  To include the commands `vexpr' and the new `csop',
  which now provides the byte string functions of the former.
  (Still available through `vexpr' until v15.)
  (I hope to be able to later provide a `usop' or `unisop' or so.)

- *v15-compat* can now have a value: if it is set, the
  `wysh' command modifier which chooses shell quoting rules
  for some commands is implicit.

- We have some (more) backward incompatible changes, though it is
  likely most users will not recognize the differences.

  o *headline* format %T is obsolete, %L fits better.

  o `csop' `hash' and `hash32' subcommands (formerly from
    `vexpr') use a slightly changed hash algorithm.
    (Which results in an improved distribution for tested sets of
    words in power-of-two spaced dictionary.)
    These are affected by the change in the second next item, too.

  o Changed to use shell quoting rules for arguments:

    + `mimetype' and `unmimetype'.
      This is affected by the change in the next item, too.

    + `shortcut' and `unshortcut'.

    + `mlist' and `unmlist' as well as `mlsubscribe'
      and `unmlsubscribe'.

    + `alias', too.

  o Changed (with legacy compat) the "@[i]" modifier prefix to
    a question-mark ?[case|..] suffix, as is known from URLs.
    We head towards direction URL syntax, now here too.

    + `if' and `elif'.
      E.g., 'wysh if "abc" ==?case "ABC"' is true, as well as is
      'wysh if 0xFFFFFFFFFFFFFFFF -eq?saturated 36#1Y2P0IJ32E8E7'.
      "==?" and "-eq?" would have been sufficient, here.
      (No unsigned mode (yet) for `if'.)

      Yes, `if' and `elif' now support `wysh', and
      see already expanded arguments, then.  No more "triggers".
      This finally makes it possible to write things like
        ? wysh if X;A;wysh elif Y;B;else;C;end
      Note 'else;C' not 'else C'.

      New operators: '-n "$VAR"' and '-z "$VAR"' work like in the
      shell, '-N varname' and '-Z varname' do not test the
      expansion but the existence of variables instead.
      Two argument forms require `wysh'.

    + `mimetype' markers have changed likewise; this also
      affects *pipe-TYPE/SUBTYPE* and *pipe-EXTENSION* (with
      legacy compatibility and -v/-d obsoletion warnings)!

        ? mimetype ?t text/x-awk  awk
        ? wysh set pipe-application/pdf='?=&?\
            trap "rm -f \"${MAILX_FILENAME_TEMPORARY}\"" EXIT;\
            trap "trap \"\" INT QUIT TERM; exit 1" INT QUIT TERM;\
            mupdf "${MAILX_FILENAME_TEMPORARY}"'

    + `vexpr' (and `csop') modifiers changed likewise.
      The case-insensitive subcommands "ifind" and "iregex" have
      been obsoleted, just use the ?[case] modifier to the regular
      function instead.

      P.S.: Thanks to Rich Felker the `regex' subcommands now
      works as desired even with empty intermediate submatches.

  o Changed address parse mode for command line arguments plus.
    This modifies decade old tradition, but results in a more
    predictable behaviour i think.  Most people will possibly even
    be surprised to see the old behaviour:

      $ </dev/null s-nail-old -dsubject ' du , de <p@p> , pp , du '
      -> To: du, de <p@p>, pp
      $ </dev/null s-nail -dsubject ' du , de <p@p> , pp , du '
      -> To: "du , de , pp , du " <p@p>

    Of course anything but perfect, our address parser is very
    complicated yet far from being acceptable.  (Dr. Werner Fink)
    This affects:

    + -b, -c and To: receivers, as above.
      (We also have a new -T receiver multiplexer, which is
      configurable in this regard, please see below for more.)

    + The -r address.
      This saw more changes: the content is no longer evaluated
      via shell expression parser (when *v15-compat* is set)..
      unless explicitly requested via the *expandaddr*
      flag "shquote".

    + *sender* variable.

    + `addrcodec' command, likewise; old:
        ? addrc e du , e <w@d> , d
        du , e <w@d> , d
        ? addrc e du , e <w@d> , d
        "du , e , d" <w@d>

    + `digmsg' and `~^' now use this parse mode fix for
      headers which need a single receiver, which is backward
      compatible but now safer since it can be fooled less easily
      (to split into a list what should be a single address, as
      shown above for `addrcodec').

      They now can also be forced to use that parse mode for To:,
      Cc:, Bcc: with a new question mark modifier "?single", here
      the word "single" is optional.

       ~^ header insert To?single: exa, <m@ple>
       ~^ header show to

- By established rules and popular demand occurrances of '^From_'
  (see *mbox-rfc4155*) will be MBOXO quoted (prefixed with
  greater-than sign '>') instead of causing a non-destructive
  encoding like 'quoted-printable' to be chosen, unless context
  (e.g., message signing) requires otherwise.
  Only with *mime-encoding*=8bit.

- We now support long "Options" -- try --long-help.

- Finally, it is possible to force sending out messages with the
  new *mime-force-sendout* variable.  If this MUA has been
  compiled with iconv(3) support it can happen that sending
  otherwise valid text messages fails because of invalid bytes
  sequences according to the locale; setting this new variable
  will avoid this; use *mime-counter-evidence* to view such
  messages nonetheless.  (Dr. Werner Fink)


- `mimeview' works again with binary formats.  (Russell Bell)

- IMAP searches via IMAP without matches no longer report a single
  match.  (Dirk-Wilhelm Peters)

- New -Y aka --cmd= option to inject commands to be executed
  when startup is completed (as opposed to the earlier -X aka
  These commands appear as if the user had typed them in.

- A new *on-history-addition* can be used to filter what
  enters the `history'.

- New "fcc" flag for *expandaddr*.  (Olav Mørkrid)

  And "domaincheck" will cause target domain comparison against
  entries in the new *expandaddr-domaincheck*.  (Olav Mørkrid)

- New *mbox-fcc-and-pcc* will write out file and pipe addresses
  as a plain RFC5322 message rather than an MBOX.  (Olav Mørkrid)

- The `errors' queue existance and size is announced via
  (Russell Bell, Martin Neitzel)

- Our MBOX parser is now truly compliant to POSIX.
  (Dr. Werner Fink)

- We follow symbolic links again when writing files.
  (Russell Bell)

- *tls-rand-file* is in fact now necessarily one of the
  optional *tls-features*.  (Mike Sharov)

- New command line option -T aka --target='FIELD: BODY'.
  FIELD can be To:, Cc:, Bcc: or Fcc:.
  The BODY is parsed as a list (just as if the given FIELD would
  be part of a template message fed in via -t), but the
  "?single" modifier suffix can be used to avoid this.
  (Dr. Werner Fink)

git(1) shortlog: Steffen Nurpmeso (277 + 9)

v14.9.11 ("Tit family enjoying a bath"), 2018-08-08

A hot summer bugfix release, but it surely brings in some new
features, like TLS fingerprinting and `digmsg' message access.

An embarassing number of bugfixes have been seen, to fix IMAP UID
handling on 32-bit hosts, UTF-8, `readall' with empty lines, rare
endless iconv(3) loops, false qsort(3)ing of addressee lists,
crashes due to false user shell quoting, acceptance of "0" port
numbers, and more.  Most of these cases now have tests.

Credits, in order of commit appearance: Paride Legovini,
Andrew Gee, Olav Mørkrid, Kevin McCarthy, Michael Dressel,
Jürgen Bruckner, Robert Elz, Rudolf Sykora, Doug McIlroy,
Gavin Troy and Jörg Schilling.
A special credit to Coverity.com again, it found a bug!
(< https://scan.coverity.com/projects/s-nail>, project number 444.)

We welcome Andrew Gee, Kevin McCarthy, Michael Dressel,
Olav Mørkrid and Jürgen Bruckner in THANKS.

NOTES, ChangeLog (packager-affine)

- I have discovered that GnuPG can --export-secret-subkey so that
  the real/full private key is no longer needed to --sign, and the
  excerpts of the private one can have a different password, too.
  So i have created a new for-signing subkey: EEC8C2FF.

  Unfortunately it is not possible to verify new signatures with
  the old public key, an update is necessary.  For example via
  or just update 1883A0DD via normal gpg(1).

- Maildir support is now optional but default via OPT_MAILDIR.
  (Paride Legovini)

- I hope for the last time there has been a change to SSL
  configuration: i have renamed all ssl* variables to tls*.
  The old ssl* ones still exist until v15, though, yet obsoletion
  warnings will be produced.
  This is so because i expect that in a not too distant future
  only the term TLS will be around.

  Also the term CipherList was falsely used, it should have been

- Most (if not all) commands which take a message list and a file
  target now use shell-style quoting.  (Before that say `copy'
  scanned backwards over "something possibly quoted", took that
  off, then treated the rest as a message list.  Now shell tokens
  are parsed starting at the front, the last is taken off, and
  anything before that is the message list.)  (Gavin Troy.  2013.)

- `~^#'? "header show" now backward-incompatibly shows the address
  type in field 1, but since this _only_ applies to non-network
  addresses i made the change.

- We will find ncurses on DragonFly BSD.

- On Solaris tests no longer need GNU cksum(1): the Solaris cksum
  is different only for whitespace separators.  (Jörg Schilling)

- All generated files reside in .obj/, and the tests run in there,
  too.  A "rm -f .obj" should suffice to clean anything up.


- `~F', `~f', `~M', `~m', `~U' and `~u'
  now default to the current message (the "dot").  (Andrew Gee)

- *indentprefix* handling has had its pitfalls when quoting
  messages.  (Andrew Gee)

- -r will again set *from* even after -S has been used
  to set *from*.  (Michael Dressel)

- No longer process From: (*from*) content via `alternates'
  when Sender: (*sender*) is set.  (Michael Dressel)

- Because the priority class of headers was not taken into
  account, it could happen that addressees in Cc: would remain but
  the same in To: were removed.  (Michael Dressel)

- IMAP accounts for RFC 4551 (and 7162) and supports 64-bit

- *spamfilter-rate-scanscore* could crash if specification
  did not match program output.

- Shims for TLSv1.3 support, e.g., for *tls-config-pairs*.

- Obsoleted *dotlock-ignore-error*, added *dotlock-disable*.
  (Paride Legovini)

- In compose-mode, removing the In-Reply-To: header breaks an old,
  and starts a new thread.  (Doug McIlroy)

- Added new *forward-inject-tail*, *quote-inject-head*
  and *quote-inject-tail* variables, and extended the meaning
  of *quote*.

  All of *{forward,quote}-inject-{head,tail}* now support
  a compose-mode specific set of formats (see
  *quote-inject-head*), for now a few only.
  (This adds meaning onto the content of *forward-inject-head*
  as introduced in v14.9.0.)

  The generated output honours *quote-fold*, which now takes
  an optional third argument in order to produce better output.

  While here, introduce the new command escape `~Q' which
  performs full *quote* cycles on the given message list.

- Fcc: headers are now understood in -t templates or when
  placed in compose mode (`~v', *editalong* etc.).
  Since each such header only takes one addressee, no quoting
  issues apply, the entire header body is the value.

- `~|' will pass the entire message including headers when
  used as "~||", e.g., prepend a file-carbon-copy message header:

    ~|| echo Fcc: /tmp/test; cat

- New `tls' multiplexer command.  Yet primitive and only
  supports a `fingerprint' subcommand.  Supports `vput'.

  The new *tls-fingerprint* variable chain aids in adding
  support for connection verification without an installed CA
  certificate pool in conjunction with the new
  *tls-fingerprint-digest* chain.

  Consequently *smime-sign-message-digest* has been renamed to
  *smime-sign-digest* (old version will cease in v15).
  The latter now defaults to SHA512 if possible.

- New MLE commands mle-go-screen-bwd and mle-go-screen-fwd to go
  backward and forward one screenful.
  And a new mle-clear-screen command.  (Todd C. Miller)

- New *expandaddr* setting "shquote" will evaluate addresses
  as if specified within $'' shell-quotes for -b, -c,
  and all direct command line receivers.  This allows for, e.g.,

    $ s-nail -Sexpandaddr=shquote '\$contact-mail'

- *quote-as-attachment* no longer needs to be set before
  compose mode is entered in order to become honoured.

- Even for -H or -L *folder-hook*s will now be called.
  Possible sorting is also applied.

- `=' now optionally supports message list arguments and the
  `vput' modifier in order to store the result list.

  The new `digmsg' multiplexer adds some message access, just
  like `~^' does in compose mode.  In fact the set of commands
  is shared, yet only in compose mode `digmsg' can change messages
  or access attachments until v15, however.  For example,

  #?0[steffen@essex nail.git]$ cat > /tmp/z.rc <<'_EOT'
  define one {
     if [ "${#}" -gt 0 ]
        digmsg create $1 - # no `read'/`readall' overlay but stdout
        #digmsg $1 header list
        digmsg $1 header show subject
        digmsg remove $1
        \eval xcall one "$@"
  define all {
     local set all # localize ("localopts yes" would do too)
     vput = all *; echo all: $all; eval call one $all
  #?0[steffen@essex nail.git]$ MAILRC=/tmp/z.rc \
    .obj/s-nail -:u -Snoheader -Squiet -Rf /tmp/z
  ? call all
  all: 1 2 3
  212 Subject
  Re: [S-mailx] FYI: after USB stick loss i have rotated keys, plus

  212 Subject
  Re: Problem with page?

  212 Subject
  Re: s-nail Source ...

  ? x

git(1) shortlog: Steffen Nurpmeso (203)

v14.9.10 ("(40th Mail anniversary) Blue tit"), 2018-03-25

On this day in 1978 Kurt Shoens placed the following comment in
def.h (now it is in nail.h):

   * Mail -- a mail program
   * Author: Kurt Shoens (UCB) March 25, 1978

v14.9.10 is mostly a stability and bugfix release.
It has seen a full test series including Coverity.com scans.
It fixes bugs i have introduced (also a double free in IMAP cache
that i introduced for v14.9.* series to address Coverity CID

In the end i am saying thanks to Gunnar Ritter for the IMAP
module, and absolutely especially his really neat idea of an IMAP
cache including offline work queue.  (IMAP will nonetheless
temporarily go in v15, but these ideas will come back thereafter.)
I have gray hairs now.

Credits, in order of commit appearance: William Yodlowsky,
Stuart Henderson, Jörg Schilling, Viktor SZÉPE, Rich Felker,
Ralph Corderoy and Philipp Gesang.

A special credit to Coverity.com again.  Because:
    tcc is 618496 bytes, pcc is 851968+24576 bytes,
    but gcc is 73355264 bytes and clang is even
    147406848 bytes, i wonder why the latter two never
    said a word that would have addressed the pretty
    obvious CID 1387053!
    [Use of initialized value, the author.]

We welcome Stuart Henderson and Philipp Gesang in THANKS.

NOTES, ChangeLog (packager-affine)

- The balls are now build with umask 0022 not 0027.
  (William Yodlowsky, Stuart Henderson)

- One actual development of this version was the addition of
  multiple choice VAL_ues, as documented in make.rc.
  For now we have VAL_IDNA (for OPT_IDNA)
    VAL_IDNA="idnkit idn2 idn"
  and VAL_RANDOM (by itself)
    VAL_RANDOM="arc4 ssl libgetrandom sysgetrandom urandom builtin"
  (Stuart Henderson)

  In brief:
    The value is interpreted as a whitespace separated list of
    strings, like "idn2 idn idnkit", case is ignored, order is
    The special strings "all" and "any" as well as the empty value
    are wildcard matches; if any entry in the list is a wildcard
    match, the rest of the list is ignored.

    The special string "error" will abort configuration once its
    list position is reached; this is only supported if
    documented, and not with an accompanying OPT_ (which then
    offers "require", as below).

  Since this VAL_RANDOM approach is so much better i have dropped
  OPT_SSL_RANDOM and OPT_NOEXTRANDOM that were recently
  introduced again.  They were c..p.

- Support for idnkit 2.3 has been added.
  Support for idnkit 1 (especially as idnkitlite) has been fixed.

- For the first time this codebase should be able to handle
  invalid MBOX mailboxes (produced by, e.g., dma(1)) gracefully.
  I hope i have found all places (sic) where code has to be fixed.
  E.g., "? copy * INVALID-MBOX" now works.
  (Smalltalk already knew objects which know what they are doing
  are for the better...  This is v15, then.)

- P.S.: the two FreeBSD test failures are noted in INSTALL.


- *asksend* will now really allow recomposing.

- `help' now supports recursive `commandalias'es, and
  command self-recursion detection now works differently, it has
  been false for something like

      commandalias x q; commandalias q echo au

  since q became expanded to `quit' (alias expansion equals
  new command word).  New behaviour: we allow equals once:

    commandalias q q; commandalias x q; x
    x -> q -> q -> quit

- *editalong* can have a value, say "set editalong=v" and it
  will startup $VISUAL not $EDITOR.

- Path separators are now normalized, thus all places, including
  MLE tab-expansion ("On terminal control and line editor"),
  can expand something like "///t*////t*".

- -E flag will not be obsoleted.
  -D flag has been reintroduced (sets *disconnected* right
  away, was not reinstantiated with the rest of the IMAP support.)

git(1) shortlog: Steffen Nurpmeso (71)

v14.9.9 ("Marsh tit savours first spring sun, II") 2018-03-06

A bugfix release.
I hope with this the fallout of the Christmas 2016 "address the
Dr. Problem workshop" has been fully resolved and thus MIME for
header address fields, even if iconv(3) is involved, been fully
restored!  We have even more tests for this now.

The release v14.9.8 was broken on big endian machines.
I will remove the v14.9.8 balls from the server by the weekend.
Sorry for the inconvenience!

Credits, in order of commit appearance: Slavko, Matej Mužila,
Rich Felker, Simon McVittie, Paride Legovini, Cág,
Predrag Punosevac.

We welcome Slavko, Matej Mužila, Rich Felker and Simon McVittie in

NOTES, ChangeLog (packager-affine)

- The v14.9.* series called *pipe-TYPE/SUBTYPE* handlers in
  display or quote mode with CR (carriage-return) bytes stripped
  because of a missing output file comparison check, which broke
  binary formats etc.  (Slavko)

- We now have native support for Libidn2.  (Matej Mužila)

- uname(1) is now hookable by setting the shell variable uname
  when calling make ("uname=MY-UNAME make config" etc.).
  (Simon McVittie)

  We no longer bake the kernel version into the binary, and
  `version' includes uname(2) output.
  (Simon McVittie, Paride Legovini)

- We now support a fallback P(seudo)R(andomNumber)G(enerator)
  initialization even if getrandom(2)/getrandom(3) has been found
  by the configuration, just like we do for "/dev/urandom" usage.
  This does not affect systems with arc4random(3) or OpenSSL
  random usage.  (David Čepelík, Simon McVittie)

  A new OPT_SSL_RANDOM make.rc variable, by default initialized to
  the value of OPT_SSL.


- `~@' list-edit behaviour in -# batch mode was broken.

- Character set names will now undergo generic normalization,
  including stripping of iconv(3) //SUFFIXes.

git(1) shortlog: Steffen Nurpmeso (33)

v14.9.7 ("Marsh tit patiently scraping bark") 2018-02-16

A maintenance release which fixes bugs and brings in features.

Credits, in order of commit appearance: Alexander Harm,
Viktor SZÉPE, Paul Eggert, Joseph Bisch, Paride Legovini,
and Peter J. Holzer.
A special credit to the disappearing mutt(1) bug tracker.
And to Gmane.org for creating gmane.mail.s-mailx.general!

Thanks Paride Legovini for becoming maintainer of the Debian port.

We welcome Joseph Bisch, Paride Legovini, and Peter J. Holzer in

NOTES, ChangeLog (packager-affine)

- The USB stick loss reported for v14.9.6 was fake news, so to
  say, the stick exists and therefore the old key is not

- We are back at Gmane.org!

- `history' has learned to be context-sensitive a bit, and
  has two new subcommands, `load' and `save'.

    This is in parts backward incompatible because it needs a new
    *history-file* format; however, the old format can be loaded
    yet compose-mode commands will not appear in compose mode no
    more.  Iirc you can start with an old format then `save' to
    the new, then replace the "d" in the first column with "c" for
    compose-mode commands which should appear correctly.

- Obsoletion warnings for variables now happen at `set'
  time instead of when used.  Running once via -v may
  be beneficial.

- The saturation modifier of `vexpr' is henceforth a prefix,
  the suffix version is obsolete (but still supported for a while).

- A network address that contains no domain-, but only a valid local
  user <name> in angle brackets will be automatically expanded to
  a valid address when *hostname* is set to a non-empty value;
  setting it to the empty value instructs us that the used *mta*
  (including builtin SMTP) will perform the necessary expansion.
  (Viktor SZÉPE)

  Note that *hostname* as well as *smtp-hostname*
  will now undergo IDNA expansion if IDNA is supported.

  And *from* and *sender* are now verified at `set'
  time, not when used.  (Viktor SZÉPE)

- The commit message in [d503bd82] is wrong, apologies to
  Paride Legovini.  The test(1) operator "-n" appeared in Seventh
  Edition UNIX, not V8 as falsely claimed.


- Our `addrcodec' parser chokes on lesser constructs.

- Presence of command-line MTA arguments without *expandargv*
  are now a hard error.  It was my fault that this was not the
  default from the very start.  (Viktor SZÉPE)

- Seen on the mutt bug tracker, we also still have had problems
  with time settings that cross 32-bit boundaries.  As that is
  in parts induced by the C standard, now implement those parts on
  our own, and be super careful in general.  (Joseph Bisch)

- The `~@' command escape did not shell-unquote the user input
  again and was thus a bit broken; message attachments also work

- Support custom headers from the command line via -C.
  And *customhdr* is verified upon `set' time.

- The simple builtin HTML viewer now supports <blockquote>
  elements, which many web mailers, most notably gmail, use for
  citation.  (Peter J. Holzer)

git(1) shortlog: Paride Legovini (1), Steffen Nurpmeso (66)

v14.9.6 ("Marsh tit abiding a snow storm"), 2017-12-05

A bugfix release which fixes four serious and three other bugs.
A few new features came in, too.

Many thanks go to Ralph Corderoy who reported an issue that was
caused by a terrible, terrible word reversal that i managed to
produce in December 2016, and which caused the v14.9.x series to
not MIME encode (non-address) content of address header fields!

Credits, in order of commit appearance: Thomas Dickey,
Andreas Baumann, Erich Eckner, Gaetan Bisson, Solar Designer, Cág,
Ivan Tham, Ralph Corderoy and Doug McIlroy.

We welcome Andreas Baumann, Erich Eckner, Solar Designer and Cág

NOTES, ChangeLog (packager-affine)

- After USB stick loss the authors OpenPGP key has been switched to

    pub   4096R/1883A0DD 2017-11-30 [expires: 2027-11-28]
          Key fingerprint =
            EE19 E1C1 F2F7 054F 8D39  54D8 3089 64B5 1883 A0DD
    uid                  Steffen Nurpmeso <steffen@sdaoden.eu>

- $TMPDIR no longer honoured for root runs.  (Solar Designer)

- *mime-encoding* defaults to quoted-printable again.  (Cág)

- We _can_ MIME encode even header fields which contain addresses.
  Thanks to Ralph Corderoy we now also _do_ so again!


- ***#336 now uses *ifs* when splitting.

- Freezing *ttycharset* via -S also survives using or
  setting any of $LC_ALL, $LC_CTYPE and $LANG during
  program startup.

- New `local' command modifier to localize changes.
  Yet supported only for `set', i.e., we have gained
  macro-local variables.

- `vexpr' now supports a BASE#number notation for integers,
  like 16#AFFE as an alternative to 0xAFFE.

  Hint: variable settings can most often use several bases, too,
  e.g., i have "set mime-counter-evidence=0b1111".

- Very simple form of *quote-chars* to adjust our knowledge of
  what actually is to be treated as a quote character.

- *mime-counter-evidence* deep inspection (bit four) has
  been improved for the sole cases of quoting or displaying
  a message.  So messages with less than 25% of control characters
  and such will now be displayed (made printable).  This is yet
  not configurable nor do we have a way to easily access a message
  with more than that.  (Doug McIlroy)

git(1) shortlog: Steffen Nurpmeso (44)

v14.9.5 ("Marsh tit engaged with a peanut"), 2017-10-21

A bugfix release which fixes two bugs which were cast in stone.
A few compatibility improvements (AlpineLinux, Solaris).
And minor features.

Apologies to Jörg Schilling, a git bug i think it was who caused
joining of changesets, losing a credit, and it had been pushed to
[master] before the problem was realized.

Credits, in order of commit appearance: Jörg Schilling,
Doug McIlroy, Random832, Nick Stoughton and Ivan Tham.

We welcome Nick Stoughton and Ivan Tham in THANKS.

NOTES, ChangeLog (packager-affine)

- New OPT_USE_PKGSYS option can be disabled to not automatically
  pick known package system paths (pkg(7), OpenCSW, schily).
  (Jörg Schilling)


- The software indeed _never_ dealt with iconv(3) output character
  set errors (as opposed to invalid input character set byte
  sequences etc.) for the main message body!
  And I have missed that when i tweaked our iconv layer a bit!

- Fixed a race condition with sigsuspend(2) that i could only see
  on OpenBSD.  config.h offers n_SIGSUSPEND_NOT_WAITPID, by the
  way, which saves some systemcalls and did not run races, but
  noone adjusts this file.

- Message list specifications gained two new colon modifiers, one
  can now "search :Ll" to find "Mailing lists".
  The *headline* format %T now also uses L and l rather than
  S and L accordingly.

  New `addrcodec' subcommand `skinlist' acts like `skin'
  but stores in *!* *^ERR*-EXIST if the address is
  one of the known "Mailing lists".

- `echo' family now supports `vput' and *!* error
  storage, offering some kind of printf(1) experience, almost.

git(1) shortlog: Steffen Nurpmeso (35)

v14.9.4 ("(5th anniversary) Marsh tit"), 2017-09-18

This is an update feature release but which also ships a furious
number of bug fixes, about six of which were pretty serious.  It
also applies overall trimming, and improves configuration time
compatibility on macOS.

Thanks to Alexander Harm there is now a macOS Homebrew package.

Credits, in order of commit appearance: Paul Vojta, Daniel Lublin,
Alexander Harm, Norman Ramsey, Viktor Szépe, Rich Salz,
David Čepelík, Ralph Corderoy, Stéphane Chazelas, Aharon Robbins,
Ken Hornstein.

We welcome Daniel Lublin, Alexander Harm, David Čepelík and
Stéphane Chazelas in THANKS.

NOTES, ChangeLog (packager-affine)

- Out-of-tree builds have become possible via the new
  make-emerge.sh script:

  $ cd /tmp && mkdir build && cd build &&
    ~/src/nail.git/make-emerge.sh &&
    make tangerine DESTDIR=.ddir

  We now have a `citron' make target which is like `tangerine' but
  does not run the tests.

- Configuration with OPT_AUTOCC honours $CC=cc. (Norman Ramsey)

- SSL/TLS configuration has been revamped (again) in order to
  support new possibilities of OpenSSL (and LibreSSL) without
  ending up and introducing more and more variables.

  Instead we now have *ssl-config-pairs*, a comma-separated
  list of all options.  With e.g. OpenSSL 1.1.xx this will be
  directly passed through to SSL_CONF_cmd(), so there _anything_
  can be passed, otherwise we use a builtin parser to map.
  The new *ssl-features*#? states what is supported.  E.g.:

    if [ "$ssl-features" =% +ctx-set-maxmin-proto ]
      wysh set ssl-config-pairs='\
      wysh set ssl-config-pairs='\
          Protocol=-ALL\,+TLSv1.1 \, +TLSv1.2'

  OpenSSL v1.1.xx also introduces an interesting and neat idea to
  centralize SSL/TLS configuration of (all) programs in a single
  file.  This can be driven via *ssl-config-file* and the new
  *ssl-config-module* variables, several entries per program
  are allowed, see *ssl-config-module* for an example.

  New manual section "Encrypted network communication".

- Variables set or unset via -S are now frozen until program
  startup is complete.


- Historical behaviour of *askcc* / *askbcc* has been
  reintroduced.  (Norman Ramsey)

  A new *asksend* variable will show a final header summary
  and allows reentering compose mode.  Set by default.

  POSIX mirrors *ask* onto *asksub*, so dropped" the former.

- `~^' no longer normalizes header names to titlecase.

- We no longer generate charset=binary MIME parameters.
  This was introduced on 2013-01-02 and was i think owed to
  file(1)s -i output as i failed to find any other reference.
  (Normal Ramsey)

- *mime-alternative-favour-rich* now also works for handlers
  installed via *pipe-TYPE/SUBTYPE*. (Viktor Szépe)

- v14.9.* series did not generate In-Reply-To: headers!

- `alias' now supports high-bit bytes and semicolon.
  Expect that at some later time the input must be valid according
  to the locale, though. (Norman Ramsey)

- Combinations of *record* could crash because of an
  unterminated variable function argument list. (Norman Ramsey)

- New command `readall' loads an entire file into a variable.

  *signature* has been obsoleted.

- `vexpr' now supports negative arguments for the substring
  subcommand and adds trim, trim-front and trim-end subcommands.

- `!' can be used in send mode.

- `~A', `~a', `~I' and `~i' will henceforth expand
  \t and \n only if *posix* is set.
  Please use `set' instead (with `wysh', until v15).

- New "The mime.types files" type marker: @q ("quiet").

git(1) shortlog (edited): Steffen (Daode) Nurpmeso (90)

v14.9.3 ("Crested tit nibbling sunflower seeds"), 2017-08-03

This is a bugfix release but which ships some improvements, too.
It silently replaces both of v14.9.1 v14.9.2 from earlier this
week, which were broken or not entirely fixed.

Credits, in order of commit appearance: Felix Fontein, Paul Vojta,
Ralph Corderoy, Christos Zoulas, Gavin Troy, Gaetan Bisson.
Thanks, Coverity.com.

We welcome Christos Zoulas in THANKS.

Apologies to Viktor Szépe for the false spelling of his name in
the v14.9.0 announcement.
And to Gaetan Bisson for not giving credit for [14fbce97]!

NOTES, ChangeLog (packager-affine)

- fakeroot support was blindly taken from Debian and broken.
  (Gavin Troy, Gaetan Bisson)

- Base64 output was broken for cases which involved iconv(3).
  The data is not lost, you can read it with S-nail v14.9.0 and
  above, and save it somewhere.  I know of no other base64 decoder
  which reads those things correctly, though.  We now have tests.
  Along this i fixed an iconv(3) error which likely caused
  stateful decoding (like, e.g., for ISO-2022-JP) to fail because
  of an unnecessary reset of the iconv(3) state machine.
  Thanks to Gaetan Bisson for mentioning this issue!


- In compose mode the MLE allows empty lines again.

- We no longer require a writable $HOME.  Due to false code
  flow (but but but: with correct comment) a non-writable $HOME
  entry in /etc/password (i.e., from getpwuid(3)) would lead to
  a crash.  (Felix Fontain; Ralph Corderoy)

- Two faulty string operations slipped into the IMAP code,
  resulting in a crash and a "is-same-host" test that would fail
  for IMAPS connections like `save' or `copy' because of an
  implicit IMAP protocol for the target of those operations (thus
  IMAP != IMAPS).  (Paul Vojta)

- The MLE tab-expansion will now automatically append a "/" if
  there is only one possible expansion and that is a directory,
  saving the user one <TAB>.  (Christos Zoulas)

  The shell expression parser had a bug regarding understood
  metacharacters (;|&), which in turn could cause an infinite loop
  in the MLE tab-expansion for, e.g., "move &9 +<TAB>", because
  the "&" would never have been stepped over.

- New `~I' command escape is like `~i' but does not append
  a newline.

- `localopts' gained an optional second argument.
  It is now possible to specify that any macro `call'ed
  will have localopts enabled, and it is possible to fixate the
  setting so that it cannot be reverted.

- *@* should now act completely compatible to the sh(1)ell,
  thus obsoleting my hysteric warnings in the v14.9.0 announcement.

- The `Lreply', `reply', `Reply' series as well as
  `mail' now manage the error status *!*.
  I.e., there are now errors like *^ERR*-DESTADDRREQ,
  ^ERR-NODATA, ^ERR-PERM and similar.  It is not perfect yet,
  because $DEAD may have been written (with *save*) or not,
  for example.

  `Lreply' and `reply' have been rewritten rather completely
  indeed.  They join Reply-To: and Mail-Followup-To: dependent on
  the context (i.e., *reply-to-honour*, *followup-to-honour*,
  see "Mailing lists" for the picture), and if they did, use
  this list as the receivers exclusively.  It now honours
  *recipients-in-cc* even for such addressees.  (And now i wonder
  whether i should have credited Paul Vojta for that.)

  Also `Lreply' would have crashed for mails with Reply-To: but
  without *reply-to-honour* set.  We now have a test.

  Note *replyto* is obsoleted in favour of *reply-to*.

v14.9.0 ("Long-tailed tit"), 2017-07-16

This is a major feature release which took about ~22 months (24
less two) of development to complete, and which imposed massive
changes under the hood, but also quite a lot of user visible
changes, including some **backward incompatibilities**.
As usual, "s-nail -d" will show obsoletion warnings.

We gain noticeable improvements regarding scriptability and its
reliability, but also for interactive use cases, especially
notable to users is our completely new M(ailx)L(ine)E(ditor) that
supports rather real tabulator expansion and program-mode-context-
sensitive key bindings.

We now support macros with arguments, which can be `shift'ed,
a `return' status can be used, and a `vexpr' multiplexer offers
some arithmetic and string operations.  `commandalias'es are
recursive, further command modifier prefixes, like `ignerr', give
a hand that we otherwise could not offer.  In compose-mode the new
`~^' command escape allows some message and attachment access, and
can be used, e.g., to implement things like custom headers, and
has been especially designed for scripted access via the new
*on-compose-splice* and *on-compose-splice-shell* hooks.

S-nail will move (more or less) backward-incompatibly to sh(1)ell
compatible argument quoting (documented in "COMMANDS"), and an
increasing number of commands do support this already: new ones
exclusively, some old ones have either been switched (like
`localopts'), others -- noticeably `set' -- can be switched to the
new syntax with a `wysh' command modifier prefix.  E.g.:

  ? define __xv {
    # Be careful to choose sh(1)ell-style on _entire_ line!
     localopts yes; wysh set verbose; ignerr eval "${@}"; return $?
  ? commandalias call echo boo-boo
  ? commandalias xv call __xv
  ? xv list
  ? commandalias xv '\'call __xv
  ? xv list

Calling the latter `xv' for `list' will give more detailed command
information, including which kind of argument is used.

I have not managed to implement the three features i have started
this development cycle for, these are thus subject to further
development, just like wysh for message-list argument commands to
support, e.g., negation, wysh for `if' and consorts, the --
terminator to finally overcome the ridiculous requirement to quote
entire shell commands filenames for commands like `pipe.
And and and.

Credits, in order of commit appearance: Antonio Radici,
Aharon Robbins, Mike Frysinger, Predrag Punosevac, Michael Convey,
Hariskar, Rudolf Sykora, Martin Neitzel, Gavin Troy,
Salvatore Bonaccorso, Todd C. Miller, Sergey Matveev, Robert Elz,
Mantas Mikulėnas, Respiranto, Jens Schleusener, Walter Alejandro
Iglesias, Ralph Corderoy, David Levine, Lyndon Nerenberg,
Thomas Dickey, Afan, Justin Ellingwood, Ingo Schwarze,
Viktor Szépe, Gaetan Bisson, Juan RP, William Yodlowsky,
Hilko Bengen, Matthew Dillon, Colin Watson, Donald Mugnai,
Stephen Isard, Jürgen Daubert, Sven Neuhaus, trondd, Ismael Bouya,
Felipe Gasper, Paul Eggert, Dr. Werner Fink, Ken Hornstein,
Noel Chiappa, Random832, Doug McIlroy, Baptiste Daroussin,
Riccardo Ductor, Pietro Cerutti, Jörg Schilling, rain1, Xin LI.

We welcome Antonio Radici, Mike Frysinger, Predrag Punosevac,
Michael Convey, Rudolf Sykora, Todd C. Miller, Robert Elz,
Jens Schleusener, Walter Alejandro Iglesias, Thomas Dickey, Afan,
Justin Ellingwood, Viktor Szépe, Juan RP, Matthew Dillon,
Colin Watson, Donald Mugnai, Sven Neuhaus, Ismael Bouya,
Felipe Gasper, Paul Eggert, Dr. Werner Fink, Ken Hornstein,
Noel Chiappa, Random832, Doug McIlroy, Baptiste Daroussin,
Riccardo Ductor, Pietro Cerutti, Jörg Schilling, rain1, and

Apologies: Sergey Matveev.
Members of the Roff community which await progress.

NOTES, ChangeLog (packager-affine)

* This release brings some backward incompatibilities, outlined
  in the following.  Most users will not be affected, and we have
  added a lot of compatibility cruft, but that will vanish in v15.
  "$ s-nail -v"!

* The configuration and build system has changed.
  Packagers have received updated package files.

  o Anything which was WANT_xy before is now OPT_xy, and
    compiled-in paths and values, like PREFIX or PAGER, have
    gained a VAL_ prefix (thus VAL_PREFIX and VAL_PAGER).

    This is _not_ true for non-persistent or environmental values,
    e.g., DESTDIR, CC, etc., and also not for the overwritable
    program variables during configuration, e.g., $awk.

    And NAIL -> VAL_MAILX, though this is still a lie.

  o The make system now needs config..build..install or
    all..install or tangerine (config..build..test..install).
    Some constants which some experts may want to fine-tune have
    been moved to config.h.  Usual adjustments+doc via make.rc.

  o The `build' phase can be parallelized by setting the $MAKEJOBS
    environment variable, e.g., "make MAKEJOBS=-j4 build".
    Note this variable is not tracked in the configuration.
    (Gaetan Bisson)

  o Unless DESTDIR is set an uninstallation script will be
    installed along with the rest (see INSTALL file for more).

  o Set the new OPT_CROSS_BUILD to avoid feature runtime tests,
    only compile- and link-availability will be tested.  (Juan RP)

  o VERBOSE is implemented straight, but must be given at
    configuration time in order to become honoured.
    (William Yodlowsky)


  o The LD_LIBRARY_PATH etc. building processes will skip any path
    which contains the string "fakeroot".  (Hilko Bengen)

  o We honour a set $SOURCE_DATE_EPOCH environment variable to
    an extend that allows reproducible tests, which is why the
    repository gained a [test-out] branch with some expected plain
    text outputs.  (reproducible-builds.org; Colin Watson)

    The new *log-prefix* variable aids in improving the
    reproducibility of error messages.

  o These are upward compatible changes.

* "make OPENSSL_API_COMPAT=0x10100000 all" should work.

* Internal and environment variables are now explicitly _defined_
  and _tracked_ after variable handling has been rewritten
  completely.  Notes:

  o This means that, e.g., "$ password=NOT_SECRET s-nail" will
    **NOT** work no more, since *password* is an internal

  o But if you do, e.g., "? set TMPDIR=~/tmp", then this will
    also be reflected in the program environment (it is an
    environment variable) and thus affect child processes.

  o Therefore we no longer have `setenv' and `unsetenv'.

  o To integrate any other environment variable transparently
    into our variable management, the new command `environ'
    needs to be used, e.g., "? environ set NEWVAR=value" or
    "? environ link EXISTINGVAR".

- -H and -L have been decoupled:
   it used to be -e -L instead!

- *NAIL_{HEAD,TAIL}* have been obsoleted in favour of
  *message-inject-head* and *message-inject-tail*.

  *NAIL_HIST{FILE,SIZE}* have been obsoleted in favour of
  *history-file* and *history-size*.

  *NAIL_EXTRA_RC* has been obsoleted in favour of

  *batch-exit-on-error* has been obsoleted by *errexit*, which
  works just like the POSIX sh(1)ell "set -e" construct; the
  `ignerr' command modifier (`-' for command escapes in compose
  mode, and see below) can be used to ignore command errors even
  then.  (This will remain even if we at some later time will
  support at least some of the sh(1) constructs which "swallow"
  failures with set -e.)

  *bsdannounce* is obsolete, the feature is integrated in
  *header* as this is much more useful.  (This is however also
  dependent upon the also new but well-known $POSIXLY_CORRECT
  <> *posix*, but that is just how it is; these affect more
  behaviour, and increasing.)

- Colour support has been changed backward in- and upward (from
  user interface side) compatibly, see the manual section
  "Coloured display".

  + New commands: `colour' and `uncolour'.
    You can define context-sensitive, terminal-capability-
    sensitive settings, e.g.:

      if terminal && [ "$features" =% +colour ]
        colour iso  view-header ft=bold,fg=magenta,bg=cyan
        colour 256  view-header ft=bold,fg=208,bg=230 subject,from
        colour mono view-header ft=bold
        colour mono view-header ft=bold,ft=reverse subject,from

  + The variable *colour-pager* defines whether colour and font
    attribute sequences should be generated when viewing something
    in $PAGER.

  + Set the variable *colour-disable* to turn colour off
    without affecting established settings.

  + It is deduced via termcap(5) (see below) whether the terminal
    supports colors, e.g., "$ s-nail -Stermcap=Co#256".
    This is also true if we don't have termcap support.

  + Support for 256-colour terminals. (Gavin Troy)

- `source' series support shell pipes if the last character
   of the "filename" ends with a vertical bar |, e.g.,

      ? source 'gpg -qd ~/.s-nailrc-private.gpg |'

- Shell pipes are also supported as targets for `move',
  `copy' etc., yet unfortunately not with via a sh(1)ell token
  parser, so that the target still has to be a single argument.

      ? copy . '| cat; echo huhu'

- Support for custom headers via the new `~^' compose-mode
  command escape and in addition, or alternatively, with the
  internal variable *customhdr*, which also can be covered by
  `localopts'.  (Sergey Matveev)

  + Support of $ORGANIZATION has been dropped.

  + Command escape `~e' supports _any_ header.

  + Command escape `~^' supports _any_ header.

- New -: command line option can be used to more easily select
  which startup files should be loaded, e.g., -:/ loads none.
  (Robert Elz)

- `account's and *folder-hook*s now have `localopts'
   enabled by default.

- A first simple form of compose-mode hooks has been implemented:
  *on-compose-enter*, *on-compose-leave* and
  *on-compose-cleanup* can be set to macros which get invoked
  at appropriate times.
  For the `resend' series there is *on-resend-enter* and
  *on-resend-cleanup*: this is very likely to change once
  true message access is possible even in this mode.

  An even more powerful mechanism is available via the also new
  *on-compose-splice* and *on-compose-splice-shell* hooks.
  These are executed in child processes and communicate with the
  parent via their standard input and output, and therefore can
  do anything and act as if they were the user.

  `localopts' are enabled and cannot be disabled (and extend
  until the message is sent).
  (Jens Schleusener, Rudolf Sykora)

    ? set on-compose-splice=ocs
    ? define ocs {
      read ver
      echo Splice protocol version is $ver
      echo '~^header list'
      read hl; vput vexpr es substring "${hl}" 0 1
      if [ "$es" != 2 ]
        echoerr 'Failed to read header list, bailing out'
        echo '~x'
      elif [ "$hl" @i!% ' cc' ]
        echo '~^header insert cc Diet is your <mirr.or>'
        read es; vput vexpr es substr "${es}" 0 1
        if [ "$es" != 2 ]
          echoerr 'Failed to insert Cc:, bailing out'; echo '~x'

- "The .netrc file"

  + gained support for comments.
    (Walter Alejandro Iglesias, Ralph Corderoy)

  + `netrc' now has a "load" subcommand.

  + the new *netrc-pipe* obsoletes OPT_AGENT and
    *agent-shell-lookup*, and can be used to load an encrypted
    .netrc file, e.g.:

      ? set netrc-lookup netrc-pipe='gpg -qd ~/.netrc.gpg'

    I.e., this is in usual .netrc syntax and thus possibly much
    nicer than saying "? source 'gpg -qd ~/.credentials.gpg |'".

- termcap(5) / terminfo(5) support has been changed backward in-
  and upward (from user interface side) compatibly, please read
  "On terminal control and line editor".

  + OPT_TERMCAP is by default enabled.
    The new, by default enabled, configuration option
    OPT_TERMCAP_VIA_TERMINFO can be used to (try to) use
    terminfo(5) instead.

  + The variable *termcap* can be used to freely define or
    override terminal capabilities, and *termcap-disable* will
    disable interaction with the chosen library, leaving only
    *termcap* in charge.

    To use the so-called ca-mode on supporting terminals,
    effectively turning S-nail into a fullscreen application,
    *termcap-ca-mode* must be set.

  + The built-in line editor has been rather completely rewritten
    to be the Mailx-Line-Editor (OPT_MLE, default yes), and
    supports wide glyphs (if possible), infinite line lengths
    (2 GB) and more.  Tabulator expansion is no longer an option
    (but needs fnmatch(3)).

  + Optionally (OPT_KEY_BINDINGS, default yes) it has become
    possible to freely define key bindings for the MLE via the new
    `bind' and `unbind' commands.  These key bindings can
    make use of termcap(5) and/or terminfo(5) names.  The MLE will
    install a set of default bindings (unless there is a set
    *line-editor-no-defaults*), more so with OPT_TERMCAP,
    i.e., try "? bind*".

    Sufficient support provided, one can now, e.g., type "p " and
    then collect the message numbers to type, scrolling forward
    and backward via key-bindings, without losing the line
    content, then commit the final line.

  + OPT_EDITLINE and OPT_READLINE support have been dropped.
    The new MLE should not miss anything.  Does it?
    Tip: in an UTF-8 locale try "? !touch /tmp/hall{,öchen}" and
    then autocomplete that: once, then ^Q, and again.

- `source' can be used in `call'ed macros.
  What sounds so innocent replaced an entire machinery and got rid
  of a brilliant idea of Kurt Shoens from the 70s, but which never
  worked with Nail/Heirloom extensions, namely macros, and in the
  right order.
  Accompanying this -X can (dig multiline arguments and can) be
  used to define macros and run them etc.  Should work:

    $ s-nail -X'define x {' -Xversion -Xx -X'}' -X'call x'
    $ s-nail -X'source \' -X'"echo version|"' -Xx

  Macros can be `undefine'd from within themselves, and re-
  `define'd.  It is still not possible to define macros
  from within macros, and/or have inner macros, not to talk
  about local scoping or anything more sophisticated such.

- -u / $LOGNAME ($USER) handling has been redefined,
  and "-u USER" is now exactly the same as "-f %USER", and
  $LOGNAME (and $USER) is actively set to the active user.  (Afan)

  $LOGNAME is POSIX standardized and henceforth used and
  preferred over $USER, which came from BSD.  (Todd C. Miller)

- In the future (at least non-message-list) argument handling will
  be changed backward-incompatibly to be sh(1)ell compatible (and
  thus POSIX standardized), see "Shell-style argument quoting".
  New commands use it already today (`bind', `colour',
  `headerpick'), some others (most importantly, `set') can
  be forced to do so via the new `wysh' command prefix, as in:

    ? wysh set message-inject-tail=$'\n--steffen'
    ? bind base $'\cA,\x61' 'echo control-A and small a'

- We now actively manage *umask*: 0077 by default, but an
  empty string will use the setting that is active upon startup.
  Just like changes to (known) environment variables, this setting
  will also be inherited by any child process.
  (Walter Alejandro Iglesias)

- Anything SENDMAIL / *sendmail*-ish has been renamed to *mta*,
  *mta-arguments*, *mta-no-default-arguments* and

  The reason is that in v15 we won't even have *smtp*: it is just
  another form of MTA, and thus obsolete by itself.
  Note that *mta-arguments* is now parsed via the shell-token
  parser, so the following ends up exactly as desired.

    ? set mta-arguments='-t -X "/tmp/my log"'

  For now we support a hack that understands a file:// URL in
  *mta*, too, but that is also the default if there is no protocol.
  E.g.: "? set mta=smtp://a:b@xy.z"

- The "spamd" *spam-interface* is obsolete.  I haven't tested
  it since my main machine died, it is error prone since it assumes
  internals of the spamassassin wire protocol, and there never was
  a speed improvement over "spamc".  (However it could react upon
  the "is-spam" state of a message, which "spamc" doesn't allow.)

- The new *inbox* variable will henceforth be looked up when
  searching for a primary system mailbox (as in "? File %"),
  followed by the usual $MAIL and compile-time defined local
  mailspool search.  (Stephen Isard, Jürgen Daubert)

- The semantic of -a and `~@' have been changed, and both
  commands now use the same syntax:

    -a file[=input-charset[#output-charset]]

- New "failinvaddr" keyword for *expandaddr*.

- We finally "can" the so-called (by myself) "Dr. Problem" (a bit):
  (Dr. Werner Fink)

    $ </dev/null s-nail -d:/ -sTrödler 'Dr. D. Iet <z@a.k>' 2>&1 |\
      grep To:
    s-nail: >>> To: "Dr. D. Iet" <z@a.k>

  This can be done via the new `addrcodec', too, note this
  supports multiple modes (and the `vput' command modifier):

    $ echo 'addrcodec e Dr. Diet <to@fu.soj> Curd' | s-nail -#:/
    "Dr. Diet Curd" <to@fu.soj>

- All commands with the string "codec" in their name use different
  argument quoting, namely none at all, please read
  "Raw data arguments for codec commands".
  This means that `urlcodec' (and `imapcodec') has
  slightly changed semantics.
  And, while here: there is a new `shcodec', too.

- We gained "Command modifiers": `\' (avoid expansion of
  `commandalias'es), `vput' (store result in variable),
  `ignerr'  (ignore an error of the following command, even
  if the new *errexit* is set), `wysh' (use shell-style

    $ echo 'vput cwd resvar;echo $resvar' | s-nail -#:/

  And the usual sh(1) stuff: `return', `shift', `eval',
  plus a `xcall' stack-avoidance optimization (to be used in
  place of a `call' which would be the last called command).
  And an "expr(1) like thing", yet simple, `vexpr'.

    $ echo 'vexpr + 1 2' | s-nail -#:/
    00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000011
    03 | 0x3 | 3
    $ echo 'vput vexpr resvar + 1 2;echo $resvar' | s-nail -#:/

  We actually start walking (*?*, *^*).

    ? vput vexpr res regex 'bananarama' 'Bana(.+)' '\$1\$0'
    ? echo $?/$^ERRNAME :$res:
    1/NODATA ::
    ? vput vexpr res iregex 'bananarama' '(.+)rama' '\$1\$0'
    ? echo $?/$^ERRNAME :$res:
    0/NONE :bananabananarama:

  The command `vpospar' can be used to manage the stack of
  positional parameters, i.e., much like "set --".
  It also offers the possibility to save and restore the stack to
  and from variables.  Etc.

  Btw., to contact the maintainer (make.rc variables

    ? echo $contact-web; eval mail $contact-mail

- `if' no longer performs automatic number conversion, we
  use the explicit -lt, -gt etc. syntax of the sh(1).
  Note: `if' will change to be almost identical to sh(1) if(1),
  so please ensure proper test bracketing, even if it is less

  Moreover, the default string comparison mode has changed to
  case-sensitive, just like in the shell.  This is because in the
  future this crux with trigger characters will vanish and `if'
  etc. will simply slurp in already expanded shell tokens, it will
  act like the shell in that respect.  We have modifiers, though,
  yet only "@i" for case-insensitivity, also for regex matches:

    i=`LC_ALL=C.utf8 s-nail -:/ -# -X '
       \define cset_test {
          \if [ "${ttycharset}" @i=% utf ]
             \echo $LC_ALL
             \xit 0
          \if [ "${#}" -gt 0 ]
             \wysh set LC_ALL=${1}
             \eval xcall cset_test "${@}"
          \xit 1
       \call cset_test C.UTF-8 POSIX.utf8 POSIX.UTF-8 \
          en_EN.utf8 en_EN.UTF-8 en_US.utf8 en_US.UTF-8
   [ $? -eq 0 ] && UTF8_LOCALE=$i

  Please note the `eval' in 'eval xcall cset_test "${@}".  This is
  a difference of S-nail/mailx and the sh(1)ell that will remain,
  as documented in "COMMANDS": whereas the shell implements
  a language and performs standardized expansions on the line
  until finally the command is called, S-nail will decide the type
  of command line parsing dependent on the seen command, and will
  then perform a single expansion.  Therefore "${@}" will expand
  to multiple arguments if $# is greater 0, but it will expand to
  the empty string otherwise, which is not furtherly expanded away
  since it is meaningless like it is in the shell: therefore $#
  will be 1 (the empty string) not 0.

- Using an explicit proto:// prefix should get you the desired
  thing apart of *newfolders*, e.g.:

    ? File maildir:///tmp/x.mdir
    ? copy * file:///tmp/x.mbox

- New variable *record-files* can be set to extend the meaning
  of *record*.  *record-resent* was there already.

- New variable *ifs* acts a bit like the sh(1)ell's $IFS for,
  e.g., the new `read' command.

  There is a `readctl' command which can be used to manage
  the active channel used by `read'.

- The `~' alias for `call' is gone.

- `mimetype' only allows specification of a single type per
  call, on the other hand no need to quote that.

- `mimeview' must now be used explicitly to look at any
  non-text MIME part, for normal display etc. purposes we only
  support "copiousoutput" MIME handlers.

- New *socks-proxy* can be used to proxy all network traffic
  over a SOCKS5 proxy.  (Gaetan Bisson)


- The manual has seen another major overhaul, all the variables
  are now documented in a single, sorted list, and many
  clarifications should have been added.  I hope it has become
  a better read.
  (Predrag Punosevac, Michael Convey, Hariskar, Rudolf Sykora,
  Respiranto, Thomas Dickey, Donald Mugnai)

- To support RFC 1524 a.k.a. .mailcap files (see below) many
  "trigger"-characters have been added for *pipe-TYPE/SUBTYPE*,
  which may (rarely) affect existing values.
  The .mailcap support itself is not yet implemented.

- *mime-counter-evidence* gained bit 4 (perform proper in-depth
  content inspection as necessary; set to 0xE for all bits).
  (Aharon Robbins)

- Maildir paths are now created recursively as necessary.
  (Justin Ellingwood)

- -M and -m options have been added to enforce a special
  send mode that will flag standard input / the given file with
  the specified / detected MIME 'Content-Type:'.  This can be used
  to directly send, e.g., HTML log output.
  (Viktor Szépe, Ralph Corderoy)

- Disallow symlinks on writable files.  Note this requires
  O_NOFOLLOW support for the operating-system-call open(2), but
  which has been standardized a long time ago.
  (Matthew Dillon)

- `retain', `ignore' etc. now differentiate in between
  From (the From: header) and From_ (the MBOX ident).

  In fact we now have a new `headerpick' command which
  is a multiplexer for all retain and ignore lists used, call it
  without arguments to see the current setting(s).
  In v15 only `headerpick' and the standard-imposed wrappers
  `retain' and `ignore' will remain, all other wrappers will
  vanish.  Regular expressions can now be used if available:

    ? headerpick
      headerpick type retain blahblahblah cc date from \
        mail-followup-to message-id openpgp reply-to subject to \
      #headerpick type ignore currently covers no fields
      #headerpick save retain currently covers no fields
      headerpick save ignore '^Original-.*$' '^X-.*$' '^DKIM.*$'
      headerpick forward retain cc date from list-id \
        mail-followup-to openpgp reply-to subject to
      #headerpick forward ignore currently covers no fields

- `top' has been rewritten completely, `Top' is new.
  It uses a built-in set of retain/ignore headers, but it is
  possible to register a custom set via `headerpick'.
  Also, *toplines* has been extended a bit and the new
  *topsqueeze* variable may pimp your `top' experience.

    ? headerpick top retain add subject
    ? top
    [-- Message  1 -- 87 lines, 4791 bytes --]:
    Subject: Re: I can't dist to myself

    I wrote:
        3.22. bounce_delivered

- `features' has been dropped, `version' extended.

- The *prompt* handling has changed: we lost the capability to
  expand \?, \@ and \$, instead new "private" variables *?*,
  *account*, *mailbox-resolved* and
  *mailbox-display* have been introduced, and the prompt
  is completely shell expanded (thus twice with `wysh' or in v15),
  as if dollar-single-quote quoted.  We do support the reverse-
  solidus escaped bracket notation for embedding characters which
  should not be counted when calculating the width of the prompt.
  The `colour' command has a slot for the prompt colour.
  We gained *prompt2* as a second level prompt.

    ? var prompt
    wysh set \
      prompt='?\${?}!\${!}[\${account}#\${mailbox-display}]? '

- The filename "-" can be used as a receiver, e.g.,

    $ echo Hey,\ you | s-nail -:/ -Sexpandaddr -sUB -

- The -s command line option, the `~s' command escape
  as well as the corresponding slots of `~^' will actively
  strip [\r\n] from their value (Debian #419840).

- New `read' and `echoerr' commands, mostly for
  But also `echon' and `echoerrn', which do not write
  a trailing newline.

- New variable *r-option-implicit* may be helpful to those
  who regulary need the functionality of the -r command
  line option.  (Felipe Gasper, Martin Neitzel)

- By using new "pseudo-URLs" one can automatize the use of S/MIME
  keys / (certificates / intermediate include certificates) with
  passwords.  E.g., to drive bob@exam.ple, set
  *smime-sign-cert-bob@exam.ple* to the private key / certificate
  pair as usual, the password lookup will then be performed for
  bob@exam.ple.smime-cert-key, bob@exam.ple.smime-cert-cert and
  Like this the password can be stored in an encrypted .netrc file
  when *netrc-lookup* and *netrc-pipe* are set, or it may
  be stored in an encrypted resource file that has been loaded via
  `source' as a simple *password* variable.

  Note that the prompting that happens as a last resort of
  password lookup will still interfere with a possibly running
  $PAGER instance, dependent on the setting of *crt*, of
  course.  Proper job control handling and recognizing that we are
  running $PAGER when doing that prompt is a TODO for v15.  Sorry.

- Some commands, like `set', `help', `list',
  `mlist' etc., now react upon the setting of *verbose*
  and(/or) *debug*.

- `write' uses iconv(3) as appropriate.

- *mbox-rfc4155* has first been dropped, and was then
  reintroduced with different semantics.  Because, it can be
  helpful if a messed up MBOX is read, in which case we henceforth
  will warn you and point you to this:

    ? define mboxfix {
      \localopts yes; \wysh set mbox-rfc4155;\
        \wysh File "${1}"; \eval copy * "${2}"
    ? call mboxfix /tmp/bad.mbox /tmp/good.mbox

  P.S.  Here you see how weird the current thing still is, in v15:

    ? define mboxfix {
      localopts yes; set mbox-rfc4155; File "${1}"; copy * -- "${2}"

  And also in v15 we will not apply (proper) so-called MBOXO
  quoting, but instead (simply MIME) re-encode mail messages.

- `call_if' is new and, different to "? ignerr call", silent
  and not messing with the return status.

- The new *smime-ca-flags* and *ssl-ca-flags*#? can be used
  to fine-tune X509_STORE_set_flags(3) a.k.a the X509 CA
  certificate verification.

    ? set ssl-ca-flags=partial-chain
    ? wysh set smime-ca-flags="${ssl-ca-flags}"

  Also, *ssl-curves*#? for TLSv1.3.

- Socket connections use TLS S(erver)N(ame)I(ndication) as
  appropriate (RFC 7817).

- `alternates' checks arguments and supports `vput'.
  It by default no longer replaces but appends alternates, unless
  *posix* mode is active.  There is a new `unalternates'
  command to remove alternates.

- A new `charsetalias' command.  (Pietro Cerutti, mutt#3925)

- New commands `filetype' and `unfiletype': in the future
  we will no longer know any builtin filetypes, in fact we already
  simulate .gz etc. via the new mechanism as necessary:

    ? filetype \
       bz2 'bzip2 -dc' 'bzip2 -zc' \
       gpg 'gpg -d' 'gpg -e' \
       gz 'gzip -dc' 'gzip -c' \
       xz 'xz -dc' 'xz -zc' \
       zst 'zstd -dc' 'zstd -19 -zc' \
       zst.pgp 'gpg -d | zstd -dc' 'zstd -19 -zc | gpg -e'

- `~<' now offers a "- [HERE-delimiter]" mode for pasting etc.
  (Ralph Corderoy)

- `exit' and `quit' take an optional exit status.
  (That is not fixated yet, though.)

- We have a useful -h / --help output.  (Doug McIlroy)

- *encoding* obsoleted in favour of new *mime-encoding*, which
  now defaults to base64.

- *allnet* now works (broken since nail 10.00, 2002-09-29).


The complete changelog of commits in between two versions OLD and
NEW can be inspected by using the git(1) `log' command:

  $ git log --reverse --topo-order --abbrev-commit OLD..NEW
  # Only topic branch headers (--no-merges for content commits only):
  $ git log --oneline --reverse --topo-order --merges OLD..NEW
  # Same, but truly accessible:
  $ git log --oneline --reverse --topo-order --merges --parents OLD..NEW |
    while read c1 c2 c3 c4 c5 c6; do
      printf "%-24s: \$ git log --oneline --no-merges %s ^%s\n" \
        "${c6}" "${c1}" "${c2}";

Entries for releases before v14.9.0 have been cut off and can be
found in the git(1) repository:

  v14.8.0 - v14.8.16: $ git show v14.8.16:NEWS
  v13     - v14.8.5 : $ git show v14.8.5:NEWS
  9.0     - 12.5    : $ git show heirloom:ChangeLog

Also accessible via HTTPS?, just replace X.Y.Z accordingly:


For even older releases you need to look into the [timeline]
branch, but no changelog has been administrated for them.




Copyright (c) 1997 - 2021, Steffen Nurpmeso <steffen@sdaoden.eu>
@(#)code-nail-ann.html-w42 1.32 2021-01-16T00:24:27+0000