From cl at isbd.net Tue Feb 5 22:10:51 2019 From: cl at isbd.net (Chris Green) Date: Tue, 5 Feb 2019 22:10:51 +0000 Subject: HTML mail which isn't handled correctly by 'auto_view text/html Message-ID: <20190205221051.GB9262@esprimo> I am getting notification E-Mails from DPD which aren't handled correctly by 'auto_view text/html' in my muttrc and 'text/html; lynx -dump %s; copiousoutput; nametemplate=%s.html' in .mailcap. Presumably it's simply that there is no 'text/html' string in the headers, I just get to see the raw HTML. In fact there is the following:- ... ... ... Subject: Your order 3179771 is due for delivery by DHL Parcel UK Content-Type: multipart/alternative; boundary="--boundary_145134_db6b4c85-2751-452f-bf4b-c56010b07587" Message-ID: Status: RO Content-Length: 20807 Lines: 285 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable ... ... ... So that would explain it! Is there any way to manually ask mutt to treat this as text/html? -- Chris Green From cs at cskk.id.au Wed Feb 6 02:00:48 2019 From: cs at cskk.id.au (Cameron Simpson) Date: Wed, 6 Feb 2019 13:00:48 +1100 Subject: HTML mail which isn't handled correctly by 'auto_view text/html In-Reply-To: <20190205221051.GB9262@esprimo> References: <20190205221051.GB9262@esprimo> Message-ID: <20190206020048.GA56907@cskk.homeip.net> On 05Feb2019 22:10, Chris Green wrote: >I am getting notification E-Mails from DPD which aren't handled >correctly by 'auto_view text/html' in my muttrc and 'text/html; lynx >-dump %s; copiousoutput; nametemplate=%s.html' in .mailcap. > >Presumably it's simply that there is no 'text/html' string in the >headers, I just get to see the raw HTML. In fact there is the >following:- > > ... > ... > ... > Subject: Your order 3179771 is due for delivery by DHL Parcel UK > Content-Type: multipart/alternative; > boundary="--boundary_145134_db6b4c85-2751-452f-bf4b-c56010b07587" > Message-ID: > > Status: RO > Content-Length: 20807 > Lines: 285 > > Content-Type: text/plain; charset="us-ascii" > Content-Transfer-Encoding: quoted-printable > > > > > > ... > ... > ... > >So that would explain it! Is there any way to manually ask mutt to >treat this as text/html? Kinda. Firstly, is that really the entirely of the relevant headers? No text/html or multipart/mixed? DHL are really sending HTML notifications (ugh!) identified as text/plain? That is a new low. (My previous low is the many senders who send multipart/mixed with either the HTML duplicated in the text/plain half or outright absent - empty text/plain half.) So: are DHL sending _only_ an text/plain section or both text/plain and text/html? In my experience it is usually the latter. Scenario A: Both text/html and text/plain, but the text/plain contains HTML. I get this a lot. I have this in my muttrc: ############################ # alternative-order criteria message-hook . 'unalternative_order *; alternative_order text/plain text/html' # Apple Mail embeds attachments in the HTML part instead of outside # the multipart/mixed message-hook '~h "X-Mailer: Apple Mail" ~X 1-' 'unalternative_order *; alternative_order text/html multipart/mixed text/plain' # senders who can't seem to master multipart/mixed, and send empty # or useless text/plain sections # or just badly badly formatted plain text, such as live.com etc message-hook '%f htmlers | ~f @no-reply at cc.yahoo-inc.com | ~f @outlook.com | ~f live.com | ~f @facebookmail.com' 'unalternative_order *; alternative_order text/html text/plain' which uses "alternative_order text/plain text/html" normally, but reverses that for some special messages: Apple Mail with attachments, and email from known bad domains and senders in my %htmlers mutt group, a list of specific know bad authors. Scenario B: If DHL really are sending HTML in a standalone text/plain body then you've got 2 choices. The first is to act when you receive the email; if you're using procmail or something similar to file you email you could match these messages and modify the Content-Type: header to say "text/html". The alternative is to match the message with mutt, and to set display_filter specially for this message. You could use your normal display_filter and use a message-hook to override it with the command from the mailcap ("lynx -dump"). Cheers, Cameron Simpson From cl at isbd.net Wed Feb 6 13:01:25 2019 From: cl at isbd.net (Chris Green) Date: Wed, 6 Feb 2019 13:01:25 +0000 Subject: HTML mail which isn't handled correctly by 'auto_view text/html In-Reply-To: <20190206020048.GA56907@cskk.homeip.net> References: <20190205221051.GB9262@esprimo> <20190206020048.GA56907@cskk.homeip.net> Message-ID: <20190206130125.GB28426@esprimo> On Wed, Feb 06, 2019 at 01:00:48PM +1100, Cameron Simpson wrote: > On 05Feb2019 22:10, Chris Green wrote: > > I am getting notification E-Mails from DPD which aren't handled > > correctly by 'auto_view text/html' in my muttrc and 'text/html; lynx > > -dump %s; copiousoutput; nametemplate=%s.html' in .mailcap. > > > > Presumably it's simply that there is no 'text/html' string in the > > headers, I just get to see the raw HTML. In fact there is the > > following:- > > > > ... > > ... > > ... > > Subject: Your order 3179771 is due for delivery by DHL Parcel UK > > Content-Type: multipart/alternative; > > boundary="--boundary_145134_db6b4c85-2751-452f-bf4b-c56010b07587" > > Message-ID: > > > > Status: RO > > Content-Length: 20807 > > Lines: 285 > > > > Content-Type: text/plain; charset="us-ascii" > > Content-Transfer-Encoding: quoted-printable > > > > > > > > > > > > ... > > ... > > ... > > > > So that would explain it! Is there any way to manually ask mutt to > > treat this as text/html? > > Kinda. > > Firstly, is that really the entirely of the relevant headers? No text/html > or multipart/mixed? DHL are really sending HTML notifications (ugh!) > identified as text/plain? That is a new low. > Yes! It's the only message I've ever received (well, I've received a couple of these from DHL recently) which has done this to me. Everyone else's HTML (spit) E-Mail gets fed into Lynx correctly. > (My previous low is the many senders who send multipart/mixed with either > the HTML duplicated in the text/plain half or outright absent - empty > text/plain half.) > > So: are DHL sending _only_ an text/plain section or both text/plain and > text/html? In my experience it is usually the latter. > There's only the HTML, identified as plain! > Scenario A: Both text/html and text/plain, but the text/plain contains HTML. > I get this a lot. I have this in my muttrc: > Not this I think. > > Scenario B: > > If DHL really are sending HTML in a standalone text/plain body then you've > got 2 choices. > > The first is to act when you receive the email; if you're using procmail or > something similar to file you email you could match these messages and > modify the Content-Type: header to say "text/html". > > The alternative is to match the message with mutt, and to set display_filter > specially for this message. You could use your normal display_filter and use > a message-hook to override it with the command from the mailcap ("lynx > -dump"). > OK, thanks, it's quite rare and, at the moment, only DHL notifications doing this that I actually want to see the content. -- Chris Green From cs at cskk.id.au Wed Feb 6 22:44:25 2019 From: cs at cskk.id.au (Cameron Simpson) Date: Thu, 7 Feb 2019 09:44:25 +1100 Subject: HTML mail which isn't handled correctly by 'auto_view text/html In-Reply-To: <20190206130125.GB28426@esprimo> References: <20190206130125.GB28426@esprimo> Message-ID: <20190206224425.GA49433@cskk.homeip.net> On 06Feb2019 13:01, Chris Green wrote: >On Wed, Feb 06, 2019 at 01:00:48PM +1100, Cameron Simpson wrote: >> Scenario B: >> If DHL really are sending HTML in a standalone text/plain body then >> you've got 2 choices. >> >> The first is to act when you receive the email; if you're using procmail or >> something similar to file you email you could match these messages and >> modify the Content-Type: header to say "text/html". >> >> The alternative is to match the message with mutt, and to set display_filter >> specially for this message. You could use your normal display_filter and use >> a message-hook to override it with the command from the mailcap ("lynx >> -dump"). >> >OK, thanks, it's quite rare and, at the moment, only DHL notifications >doing this that I actually want to see the content. Just further to the display_filter thing: I normally use a display_filter, set to a personal script which goes: #!/bin/sh # # Mutt display filter, whose behaviour adjusts accoridng to some flags. # - Cameron Simpson 29jan2017 # if flag MUTT_ROT13 then tr '[A-Z][a-z]' '[N-Z][A-M][n-z][a-m]' else cat fi \ | if flag MUTT_UNTOPPOST then untoppost else cat fi \ | mutt-highlight The mutt-highlight is a sed script which turns *foo* into bold and _foo_ into underlined. But the important thing here is the if statement: it optionally runs the message body through some filters. For example, I've got a mutt macro ^X to toggle the MUTT_ROT13 flag and redisplay the message. You could adopt such a scheme for your HTML issue (in fact, I'm going to do that myself too): have an optional flag to decode the HTML using a pipeline flavour of your mailcap unhtml line. Then bind a mutt keystroke to toggle the flag and redisplay. My rot13 bindings go: set my_toggle_rot13="$my_push_wait_key=no

flag ! MUTT_ROT13 -e flag MUTT_ROT13$my_pop_wait_key" macro index \Cx "$my_toggle_rot13" 'toggle MUTT_ROT13' macro pager \Cx "$my_toggle_rot13" 'toggle MUTT_ROT13' Just a thought. Source for any of the above is available. Cheers, Cameron Simpson From cl at isbd.net Thu Feb 7 08:36:19 2019 From: cl at isbd.net (Chris Green) Date: Thu, 7 Feb 2019 08:36:19 +0000 Subject: HTML mail which isn't handled correctly by 'auto_view text/html In-Reply-To: <20190206224425.GA49433@cskk.homeip.net> References: <20190206130125.GB28426@esprimo> <20190206224425.GA49433@cskk.homeip.net> Message-ID: <20190207083619.GA31300@esprimo> On Thu, Feb 07, 2019 at 09:44:25AM +1100, Cameron Simpson wrote: > On 06Feb2019 13:01, Chris Green wrote: > > On Wed, Feb 06, 2019 at 01:00:48PM +1100, Cameron Simpson wrote: > > > Scenario B: > > > If DHL really are sending HTML in a standalone text/plain body then > > > you've got 2 choices. > > > > > > The first is to act when you receive the email; if you're using procmail or > > > something similar to file you email you could match these messages and > > > modify the Content-Type: header to say "text/html". > > > > > > The alternative is to match the message with mutt, and to set display_filter > > > specially for this message. You could use your normal display_filter and use > > > a message-hook to override it with the command from the mailcap ("lynx > > > -dump"). > > > > > OK, thanks, it's quite rare and, at the moment, only DHL notifications > > doing this that I actually want to see the content. > > Just further to the display_filter thing: I normally use a display_filter, > set to a personal script which goes: > > #!/bin/sh > # > # Mutt display filter, whose behaviour adjusts accoridng to some flags. > # - Cameron Simpson 29jan2017 > # > > if flag MUTT_ROT13 > then tr '[A-Z][a-z]' '[N-Z][A-M][n-z][a-m]' > else cat > fi \ > | if flag MUTT_UNTOPPOST > then untoppost > else cat > fi \ > | mutt-highlight > > The mutt-highlight is a sed script which turns *foo* into bold and _foo_ > into underlined. But the important thing here is the if statement: it > optionally runs the message body through some filters. For example, I've got > a mutt macro ^X to toggle the MUTT_ROT13 flag and redisplay the message. > > You could adopt such a scheme for your HTML issue (in fact, I'm going to do > that myself too): have an optional flag to decode the HTML using a pipeline > flavour of your mailcap unhtml line. Then bind a mutt keystroke to toggle > the flag and redisplay. My rot13 bindings go: > > set my_toggle_rot13="$my_push_wait_key=no

flag ! MUTT_ROT13 > -e flag MUTT_ROT13$my_pop_wait_key" > macro index \Cx "$my_toggle_rot13" 'toggle MUTT_ROT13' > macro pager \Cx "$my_toggle_rot13" 'toggle MUTT_ROT13' > > Just a thought. Source for any of the above is available. > That's just the sort of thing I was hoping for - "oh, look, there's some raw HTML...", a couple of keystrokes and it's converted. I may well come back in a few days and ask to see the source for untoppost, the rest I can probably fathom out for myself. Thanks. -- Chris Green From invalid at pizzashack.org Fri Feb 8 01:50:36 2019 From: invalid at pizzashack.org (Derek Martin) Date: Thu, 7 Feb 2019 19:50:36 -0600 Subject: HTML mail which isn't handled correctly by 'auto_view text/html In-Reply-To: <20190205221051.GB9262@esprimo> References: <20190205221051.GB9262@esprimo> Message-ID: <20190208015036.GB22306@bladeshadow.org> Chris, Based on the excerpt you posted: On Tue, Feb 05, 2019 at 10:10:51PM +0000, Chris Green wrote: > ... > ... > ... > Subject: Your order 3179771 is due for delivery by DHL Parcel UK > Content-Type: multipart/alternative; > boundary="--boundary_145134_db6b4c85-2751-452f-bf4b-c56010b07587" > Message-ID: > > Status: RO > Content-Length: 20807 > Lines: 285 > > Content-Type: text/plain; charset="us-ascii" > Content-Transfer-Encoding: quoted-printable > > > > > > ... > ... > ... > ...it appears this is a multi-part message, theoretically intended to have both an HTML part and a plain text part, where the first part is HTML that is idiotically marked as plain text, because their IT folks are too lazy/stupid to produce a proper multipart/alternative message. I would encourage you to complain to DHL, but I would also expect that to be futile, so I won't bother. ;-) But the next thing I would recommend is look again at the message parts (press v on the message in the index). There may be an actual HTML part that you're not seeing, because your settings prefer the plain text (as mine do). In that case, Mutt is displaying the faux plain text, since you quite reasonably told it to, and doing it verbatim and not trying to run any conversion on it, since... DHL told Mutt it shouldn't need to. Probably you can select the HTML and manually display it. Either that or DHL is just really that bad at e-mail. :-( -- Derek D. Martin http://www.pizzashack.org/ GPG Key ID: 0xDFBEAD02 -=-=-=-=- This message is posted from an invalid address. Replying to it will result in undeliverable mail due to spam prevention. Sorry for the inconvenience. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available URL: From cl at isbd.net Fri Feb 8 09:49:17 2019 From: cl at isbd.net (Chris Green) Date: Fri, 8 Feb 2019 09:49:17 +0000 Subject: HTML mail which isn't handled correctly by 'auto_view text/html In-Reply-To: <20190208015036.GB22306@bladeshadow.org> References: <20190205221051.GB9262@esprimo> <20190208015036.GB22306@bladeshadow.org> Message-ID: <20190208094917.GC19106@esprimo> On Thu, Feb 07, 2019 at 07:50:36PM -0600, Derek Martin wrote: > Chris, > > Based on the excerpt you posted: > > On Tue, Feb 05, 2019 at 10:10:51PM +0000, Chris Green wrote: > > ... > > ... > > ... > > Subject: Your order 3179771 is due for delivery by DHL Parcel UK > > Content-Type: multipart/alternative; > > boundary="--boundary_145134_db6b4c85-2751-452f-bf4b-c56010b07587" > > Message-ID: > > > > Status: RO > > Content-Length: 20807 > > Lines: 285 > > > > Content-Type: text/plain; charset="us-ascii" > > Content-Transfer-Encoding: quoted-printable > > > > > > > > > > > > ... > > ... > > ... > > > > ...it appears this is a multi-part message, theoretically intended to > have both an HTML part and a plain text part, where the first part is > HTML that is idiotically marked as plain text, because their IT folks > are too lazy/stupid to produce a proper multipart/alternative message. > I would encourage you to complain to DHL, but I would also expect that > to be futile, so I won't bother. ;-) > > But the next thing I would recommend is look again at the message > parts (press v on the message in the index). There may be an actual > HTML part that you're not seeing, because your settings prefer the > plain text (as mine do). In that case, Mutt is displaying the faux > plain text, since you quite reasonably told it to, and doing it > verbatim and not trying to run any conversion on it, since... DHL told > Mutt it shouldn't need to. > > Probably you can select the HTML and manually display it. Either that > or DHL is just really that bad at e-mail. :-( > I'm pretty sure that there was only one part, I saved the message and opened it with vi[le] and a quick scan through it showed the HTML header stuff at the top and closing HTML at the end. I will check more carefully next time I get one though, thanks. -- Chris Green