sfeed, branch HEADRSS and Atom parser
ba641e578512569d70f74c9b1b309d3247ea0d142025-12-29T12:00:48Z2025-12-29T12:00:48Zbump version to 2.3Hiltjo Posthumahiltjo@codemadness.orgcommit ba641e578512569d70f74c9b1b309d3247ea0d14
parent e2537f08dde958c02f835229ae11c5db785abd2a
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Mon, 29 Dec 2025 13:00:48 +0100
bump version to 2.3
e2537f08dde958c02f835229ae11c5db785abd2a2025-12-26T20:03:27Z2025-12-27T12:26:43Zsfeed_mbox: set the author as the display name for the From header againHiltjo Posthumahiltjo@codemadness.orgcommit e2537f08dde958c02f835229ae11c5db785abd2a
parent a16377b91f996a3cc9772c23cc8aadd48a13651c
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Fri, 26 Dec 2025 21:03:27 +0100
sfeed_mbox: set the author as the display name for the From header again
SFEED_MBOX_FROM now always overrides the header if it is set.
Otherwise the default e-mail is <anonymous@localhost>, if there is an author
name it is displayed.
The name is quoted. For simplicity some characters like quotes, escape
characters or text-encoding characters are removed, they are not handled at all
(note that the author field cannot contain escape characters already, unless
the user modified the parsed sfeed(5) format).
a16377b91f996a3cc9772c23cc8aadd48a13651c2025-12-11T19:21:38Z2025-12-11T19:21:38Zxml.c: more strictly check numeric entities from the "library" sideHiltjo Posthumahiltjo@codemadness.orgcommit a16377b91f996a3cc9772c23cc8aadd48a13651c
parent 8cf6e3f18ddfaa0d71a3d0a5535a175502b1d276
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Thu, 11 Dec 2025 20:21:38 +0100
xml.c: more strictly check numeric entities from the "library" side
For sfeed there is no functional difference, because of how entities are parsed
in a certain way.
The xml.{c,h} code is reusable and some programs might use xml_entitytostr()
with less validation beforehand.
These cases are now tested in the sfeed_tests repository also (98% to 100%
coverage for xml.c).
8cf6e3f18ddfaa0d71a3d0a5535a175502b1d2762025-12-06T11:34:19Z2025-12-07T12:46:45ZREADME: tweak text: reorder alternative line directly below lynxHiltjo Posthumahiltjo@codemadness.orgcommit 8cf6e3f18ddfaa0d71a3d0a5535a175502b1d276
parent 7f6fed2b65d7686e11c7e4ccfb058b55b1401cf0
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Sat, 6 Dec 2025 12:34:19 +0100
README: tweak text: reorder alternative line directly below lynx
7f6fed2b65d7686e11c7e4ccfb058b55b1401cf02025-12-06T11:28:39Z2025-12-07T12:46:45Zbump LICENSE yearHiltjo Posthumahiltjo@codemadness.orgcommit 7f6fed2b65d7686e11c7e4ccfb058b55b1401cf0
parent 6a6eb12cb0049eb6198b6dfc78aa5f54a742cf57
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Sat, 6 Dec 2025 12:28:39 +0100
bump LICENSE year
6a6eb12cb0049eb6198b6dfc78aa5f54a742cf572025-12-01T20:34:21Z2025-12-07T12:46:45Zsfeed: remove undefined behaviour of right shifting a negative valueHiltjo Posthumahiltjo@codemadness.orgcommit 6a6eb12cb0049eb6198b6dfc78aa5f54a742cf57
parent b9305cb278e35dc274264824143aa629dda9fe95
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Mon, 1 Dec 2025 21:34:21 +0100
sfeed: remove undefined behaviour of right shifting a negative value
Found by cppcheck.
This rewrites the bitshift hack in a more clear and defined behaviour.
- improve comment for datetounix(): it does no validation of input ranges here
and expects (pre-validated) valid input as if from struct tm fields.
- micro-optimization, -68 is not needed. This is the same for "& 3" also.
b9305cb278e35dc274264824143aa629dda9fe952025-12-01T20:27:51Z2025-12-07T12:46:45Zxml.c: add comment for unicode surrogate rangeHiltjo Posthumahiltjo@codemadness.orgcommit b9305cb278e35dc274264824143aa629dda9fe95
parent bfa2642d1e469839d55060e820cf0073cabaf270
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Mon, 1 Dec 2025 21:27:51 +0100
xml.c: add comment for unicode surrogate range
bfa2642d1e469839d55060e820cf0073cabaf2702025-11-16T21:04:42Z2025-12-07T12:46:45Zxml.c: validate numeric entities more strictlyHiltjo Posthumahiltjo@codemadness.orgcommit bfa2642d1e469839d55060e820cf0073cabaf270
parent cbd70903033be0b9f3a9790ef6c61fb3f5769e66
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Sun, 16 Nov 2025 22:04:42 +0100
xml.c: validate numeric entities more strictly
validate numeric entities more strictly before passing them to strtol().
Examples:
- "&# 65;" (spaces).
- ""&#-65;" (negative number).
- "&#af;" (hex but no hexadecimal x prefix): "¯" (correct)
cbd70903033be0b9f3a9790ef6c61fb3f5769e662025-11-13T12:56:22Z2025-12-07T12:46:45Zsfeed_curses.c: remove unneeded codeHiltjo Posthumahiltjo@codemadness.orgcommit cbd70903033be0b9f3a9790ef6c61fb3f5769e66
parent 3e854e2a0d57f90bf6f7482cecb2741043ccaa6b
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Thu, 13 Nov 2025 13:56:22 +0100
sfeed_curses.c: remove unneeded code
The address of struct item is not changed so this code does nothing.
Found by cppcheck.
3e854e2a0d57f90bf6f7482cecb2741043ccaa6b2025-11-13T12:45:37Z2025-12-07T12:46:45Zminicurses.h: portability: Casting non-zero decimal integer literal to pointerHiltjo Posthumahiltjo@codemadness.orgcommit 3e854e2a0d57f90bf6f7482cecb2741043ccaa6b
parent 03b52640efbd9f8fdd2297740b1c79fa0a1a7652
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Thu, 13 Nov 2025 13:45:37 +0100
minicurses.h: portability: Casting non-zero decimal integer literal to pointer
Fixes cppcheck warning.
Checked against NetBSD curses and ncurses:
ncurses has: libcurses/term.h:#define eat_newline_glitch CUR Booleans[4]
NetBSD curses:
#define t_eat_newline_glitch(t) (t)->flags[TICODE_xenl]
libterminfo/term.h:#define eat_newline_glitch t_eat_newline_glitch(cur_term)
where flags is of type: signed char *flags;
So this type is different for minicurses, but should be fine for sfeed_curses.
03b52640efbd9f8fdd2297740b1c79fa0a1a76522025-11-11T19:50:06Z2025-12-07T12:46:40Zsfeed_mbox: add mail headers and other improvementsHiltjo Posthumahiltjo@codemadness.orgcommit 03b52640efbd9f8fdd2297740b1c79fa0a1a7652
parent 839a4563e809916ccdd8ebeac1461d8c7ffc62e7
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Tue, 11 Nov 2025 20:50:06 +0100
sfeed_mbox: add mail headers and other improvements
Note: these changes may cause duplicate messages for existing users in some
use-cases because the Message-ID, From and To header will be different.
Changes:
Add mail headers, they are now only set headers when they have a value and are
non-empty.
These headers are used by rss2email also:
- X-RSS-ID: the item ID.
- X-RSS-TAGS: the category tags, '|' in sfeed(5) is replaced by ','.
- X-RSS-URL: the item link / URL.
Also added:
- X-RSS-Author: the item author.
- X-RSS-Enclosure: the first enclosure.
This allows more convenient filtering in many mail clients.
One can further manipulate the headers using mail tools or awk or a sed
expression for example. For example changing X-Feedname to X-Label etc.
- Change the default From and To header. It does not use the system hostname or
$USER anymore.
Add environment variables with defaults:
$SFEED_MBOX_FROM: <anonymous@localhost>
$SFEED_MBOX_TO: <anonymous@localhost>
- Fixes: The To header could contain characters that are invalid for the To
header as well.
- Fixes: The Message-ID field could contain spaces or invalid characters. Now
it is "@newsfeed.local".
839a4563e809916ccdd8ebeac1461d8c7ffc62e72025-11-11T19:49:39Z2025-11-11T23:28:14Zhandle unlikely case argc < 1Hiltjo Posthumahiltjo@codemadness.orgcommit 839a4563e809916ccdd8ebeac1461d8c7ffc62e7
parent 67179090976e460a85c14ce701c0c0f0dd8af2fd
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Tue, 11 Nov 2025 20:49:39 +0100
handle unlikely case argc < 1
67179090976e460a85c14ce701c0c0f0dd8af2fd2025-11-11T19:47:26Z2025-11-11T19:55:03Zsfeed_gopher: improve output of the index fileHiltjo Posthumahiltjo@codemadness.orgcommit 67179090976e460a85c14ce701c0c0f0dd8af2fd
parent 86476894e7cc6e41856303957889cf248f4a28a2
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Tue, 11 Nov 2025 20:47:26 +0100
sfeed_gopher: improve output of the index file
- Prefix feeds with new items with "N " at the start of the line.
- Right align item counts, similar to the (left) feed sidebar in sfeed_curses.
- Move the colw() function which is used to count the column width in
sfeed_curses to util.
86476894e7cc6e41856303957889cf248f4a28a22025-11-11T19:46:04Z2025-11-11T19:46:04Zsfeed.c: fix a typo in a commentHiltjo Posthumahiltjo@codemadness.orgcommit 86476894e7cc6e41856303957889cf248f4a28a2
parent d9c6a516765d10d0548f7cbf31caa0b37ce8c262
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Tue, 11 Nov 2025 20:46:04 +0100
sfeed.c: fix a typo in a comment
d9c6a516765d10d0548f7cbf31caa0b37ce8c2622025-09-21T12:43:46Z2025-09-21T12:43:46Zsfeed_gopher: improve some comments and gopher -> GopherHiltjo Posthumahiltjo@codemadness.orgcommit d9c6a516765d10d0548f7cbf31caa0b37ce8c262
parent 558184631531ba033e2dc17ca928ca4d9beebb66
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Sun, 21 Sep 2025 14:43:46 +0200
sfeed_gopher: improve some comments and gopher -> Gopher
558184631531ba033e2dc17ca928ca4d9beebb662025-09-21T12:29:01Z2025-09-21T12:29:01Zsfeed_gopher: add some comments and simplify printing non-gopher URLsHiltjo Posthumahiltjo@codemadness.orgcommit 558184631531ba033e2dc17ca928ca4d9beebb66
parent 416c898b677fbf56161623ca1eab6fed6e614a4a
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Sun, 21 Sep 2025 14:29:01 +0200
sfeed_gopher: add some comments and simplify printing non-gopher URLs
If the item link is not a gopher:// URL then just use the text from sfeed(5)
FeedLink field (do not format the parsed version).
416c898b677fbf56161623ca1eab6fed6e614a4a2025-09-21T12:23:39Z2025-09-21T12:23:39ZREADME: update link to hurl webpage, add link to webdump (HTML to plain converter)Hiltjo Posthumahiltjo@codemadness.orgcommit 416c898b677fbf56161623ca1eab6fed6e614a4a
parent 491b8106aad5e1e0bc5e8d8e3fb6a3cd5abfd34d
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Sun, 21 Sep 2025 14:23:39 +0200
README: update link to hurl webpage, add link to webdump (HTML to plain converter)
491b8106aad5e1e0bc5e8d8e3fb6a3cd5abfd34d2025-09-21T12:22:39Z2025-09-21T12:22:39Zstyle.css: remove not needed "px"Hiltjo Posthumahiltjo@codemadness.orgcommit 491b8106aad5e1e0bc5e8d8e3fb6a3cd5abfd34d
parent 8285ff3dca6560253dfd1b710643f878a940a164
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Sun, 21 Sep 2025 14:22:39 +0200
style.css: remove not needed "px"
Similar to the other attributes with 0 values.
8285ff3dca6560253dfd1b710643f878a940a1642025-08-08T12:34:59Z2025-08-08T12:34:59Zsfeed_update/sfeedrc example: use $basename variable nameHiltjo Posthumahiltjo@codemadness.orgcommit 8285ff3dca6560253dfd1b710643f878a940a164
parent e77e0165d3151ebe763a602adaa486af2467e6df
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Fri, 8 Aug 2025 14:34:59 +0200
sfeed_update/sfeedrc example: use $basename variable name
$filename happened to work, but it is set in the global scope.
Use the intended $basename in the scope of the fetch() function.
Found while testing the fetch() function by using SFEED_UPDATE_INCLUDE=1
(so $filename was not set here).
e77e0165d3151ebe763a602adaa486af2467e6df2025-08-02T12:06:01Z2025-08-02T12:06:01Zsfeed_update: merge: change the default to use the last recordHiltjo Posthumahiltjo@codemadness.orgcommit e77e0165d3151ebe763a602adaa486af2467e6df
parent 66c4192ad01bc01c65491f30f746cce00a5dab8d
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Sat, 2 Aug 2025 14:06:01 +0200
sfeed_update: merge: change the default to use the last record
In many cases it makes sense to use the last record.
Most implementations using sort -u use a stable sort and use the first record
found by the specified key (where the key are the fields: id, title, link).
This means it kept the record found in the old file and not update the last
content.
Swap the default to use the last record instead of the first.
An alternative in awk:
merge() {
LC_ALL=C awk -F '\t' '!x[$6 "\t" $2 "\t" $3]++' "$3" "$2"
}
Thanks Ivo Cicchese for the feedback!
66c4192ad01bc01c65491f30f746cce00a5dab8d2025-03-11T20:03:43Z2025-03-11T20:03:43Zbump version to 2.2Hiltjo Posthumahiltjo@codemadness.orgcommit 66c4192ad01bc01c65491f30f746cce00a5dab8d
parent a0e39f9c7adc120921fab28da6d88a4857a70547
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Tue, 11 Mar 2025 21:03:43 +0100
bump version to 2.2
a0e39f9c7adc120921fab28da6d88a4857a705472025-02-09T11:07:00Z2025-03-07T12:36:33ZREADME, sfeedrc.5: improve If-Modified-Since exampleHiltjo Posthumahiltjo@codemadness.orgcommit a0e39f9c7adc120921fab28da6d88a4857a70547
parent f6667bb506735310994c5b90eb9abec49f03a9e1
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Sun, 9 Feb 2025 12:07:00 +0100
README, sfeedrc.5: improve If-Modified-Since example
Improvements:
- Use the exact server timestamp to compare modified content (curl -R option).
Using the client timestamp is imprecise.
- Do not use the modified timestamp of the ETag for If-Modified-Since this can
differ as well.
Probably too obvious to mention but this of course assumes the filesystem can be used
to store the modification timestamp for the modified content.
Based on feedback from free_electrician, thanks!
f6667bb506735310994c5b90eb9abec49f03a9e12025-01-07T18:55:18Z2025-02-09T11:18:23Zbump LICENSEHiltjo Posthumahiltjo@codemadness.orgcommit f6667bb506735310994c5b90eb9abec49f03a9e1
parent 4d445d3e47de2d3e5b101c1a6520690d306a5d6c
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Tue, 7 Jan 2025 19:55:18 +0100
bump LICENSE
4d445d3e47de2d3e5b101c1a6520690d306a5d6c2025-01-07T18:51:15Z2025-02-09T11:18:15Zsfeed.c: optimize tag lookup by reverting the binary search to a linear lookupHiltjo Posthumahiltjo@codemadness.orgcommit 4d445d3e47de2d3e5b101c1a6520690d306a5d6c
parent c2231c4b4f91a86df6192c9830235f3ca499810e
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Tue, 7 Jan 2025 19:51:15 +0100
sfeed.c: optimize tag lookup by reverting the binary search to a linear lookup
This removes an intended optimization.
It was changed to bsearch() in commit 3031d855807de20fc86cd3dd2375ce9473e86947
After some more testing: because the tags table is so small doing a linear scan
over the table by first checking the length, and then (if needed) the tag name
comparison is bit faster (6-11%), but of course depends on the input) in common
cases than a binary search.
In practise there is no noticable difference though.
This simplifies the code a bit.
c2231c4b4f91a86df6192c9830235f3ca499810e2024-12-26T10:53:19Z2024-12-26T10:53:19Zbump version to 2.1Hiltjo Posthumahiltjo@codemadness.orgcommit c2231c4b4f91a86df6192c9830235f3ca499810e
parent 34b0cefd3e144744d65ad1d2ad8e73c8d185b1f0
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Thu, 26 Dec 2024 11:53:19 +0100
bump version to 2.1
34b0cefd3e144744d65ad1d2ad8e73c8d185b1f02024-12-22T13:11:05Z2024-12-22T13:21:41ZREADME: document old behaviour and alternative for xargs -P and -0Hiltjo Posthumahiltjo@codemadness.orgcommit 34b0cefd3e144744d65ad1d2ad8e73c8d185b1f0
parent 7f27e5c39dcd64b4c7951e4ff8e31d98a3c12047
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Sun, 22 Dec 2024 14:11:05 +0100
README: document old behaviour and alternative for xargs -P and -0
7f27e5c39dcd64b4c7951e4ff8e31d98a3c120472024-12-06T12:17:23Z2024-12-06T12:17:23Zsfeed_update: add comment for IFSHiltjo Posthumahiltjo@codemadness.orgcommit 7f27e5c39dcd64b4c7951e4ff8e31d98a3c12047
parent 6dcce9a9e8e720aca89cf50e5f5def3ea9d57ca2
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Fri, 6 Dec 2024 13:17:23 +0100
sfeed_update: add comment for IFS
It may not be clearly visible in some editors.
6dcce9a9e8e720aca89cf50e5f5def3ea9d57ca22024-12-06T11:53:59Z2024-12-06T11:55:34Zsfeed_update: leave IFS unchanged for sfeedrc codeJeff Huffmantejing@tejing.comcommit 6dcce9a9e8e720aca89cf50e5f5def3ea9d57ca2
parent 195cbb7aa51b7726c9a3cc0f5bcd5e34968a2907
Author: Jeff Huffman <tejing@tejing.com>
Date: Fri, 6 Dec 2024 12:53:59 +0100
sfeed_update: leave IFS unchanged for sfeedrc code
IFS affects the word-splitting of unquoted variable expansions, which can cause
bugs that are very difficult to track down without in-depth shell knowledge.
Example of affected code:
prog ${var:+--opt "$var"}
195cbb7aa51b7726c9a3cc0f5bcd5e34968a29072024-11-20T18:59:32Z2024-11-20T19:50:30Zsfeed_update: separate the logic of running feeds in a functionHiltjo Posthumahiltjo@codemadness.orgcommit 195cbb7aa51b7726c9a3cc0f5bcd5e34968a2907
parent a57e31efc26780e2079d336ea46c7e0776ec42e2
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Wed, 20 Nov 2024 19:59:32 +0100
sfeed_update: separate the logic of running feeds in a function
This is not documented as an overridable function (for now).
It can be used to more easily replace the logic of running feeds with xargs
with something else like GNU/parallel.
It can be used to reproduce the previous behaviour which didn't require xargs.
See commit for the old logic: cdb8f7feb135adf6f18e389b4bbf47886089474a
Example:
feed() {
# wait until ${maxjobs} are finished: will stall the queue if an item
# is slow, but it is portable.
[ ${signo} -ne 0 ] && return
[ $((curjobs % maxjobs)) -eq 0 ] && wait
[ ${signo} -ne 0 ] && return
curjobs=$((curjobs + 1))
_feed "$@" &
}
runfeeds() {
# job counter.
curjobs=0
# fetch feeds specified in config file.
feeds
# wait till all feeds are fetched (concurrently).
[ ${signo} -eq 0 ] && wait
}
a57e31efc26780e2079d336ea46c7e0776ec42e22024-11-20T18:59:15Z2024-11-20T19:50:30Zsfeed.c: support , for decimal fractionsHiltjo Posthumahiltjo@codemadness.orgcommit a57e31efc26780e2079d336ea46c7e0776ec42e2
parent 5f373b2b7aed3f569dc97c2c42aab2aa6f3324bf
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Wed, 20 Nov 2024 19:59:15 +0100
sfeed.c: support , for decimal fractions
- ISO8601 allows a comma for decimal fractions.
- RFC3339 specifies "." (but also references ISO8601).
In particular this fixes parsing timezones after decimal fractions:
1985-04-12 23:20:50,5+04:00
Would ignore the timezone, but is now supported.
5f373b2b7aed3f569dc97c2c42aab2aa6f3324bf2024-11-20T18:59:03Z2024-11-20T19:50:30Zsfeed.c: support / as a year/month/day separatorHiltjo Posthumahiltjo@codemadness.orgcommit 5f373b2b7aed3f569dc97c2c42aab2aa6f3324bf
parent c658cf92deeca62991057d1e4e2fb8b94a2d64df
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Wed, 20 Nov 2024 19:59:03 +0100
sfeed.c: support / as a year/month/day separator
Using this separator is not in the standard and actually incorrect, because ISO
8601-1 specifies this for date periods/ranges.
However since date ranges are not supported in RSS/Atom feeds anyway and do not
make sense this separator can be supported without breaking anything.
If an ISO8601 date range would be used it will still use the first parsed date.
c658cf92deeca62991057d1e4e2fb8b94a2d64df2024-11-20T19:39:03Z2024-11-20T19:50:23Zsfeedrc.5: fix consistency for feed name argumentHiltjo Posthumahiltjo@codemadness.orgcommit c658cf92deeca62991057d1e4e2fb8b94a2d64df
parent 3f9e25899c6c860d622c18af1b7058135c4a802d
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Wed, 20 Nov 2024 20:39:03 +0100
sfeedrc.5: fix consistency for feed name argument
3f9e25899c6c860d622c18af1b7058135c4a802d2024-11-20T18:58:46Z2024-11-20T19:50:23Zcode-style: improve some code-style for multiline commentsHiltjo Posthumahiltjo@codemadness.orgcommit 3f9e25899c6c860d622c18af1b7058135c4a802d
parent b2b2426f7742a8a640f1bd8bb9709452baebdca7
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Wed, 20 Nov 2024 19:58:46 +0100
code-style: improve some code-style for multiline comments
Just noticed it when reviewing some code. It is not important though and no
promises this will be kept consistent in the future.
b2b2426f7742a8a640f1bd8bb9709452baebdca72024-11-02T11:31:26Z2024-11-02T11:31:26Zutil.h: add a note about __dead and noreturn hintsHiltjo Posthumahiltjo@codemadness.orgcommit b2b2426f7742a8a640f1bd8bb9709452baebdca7
parent 1c9433f80c0209c304236f84b58f87bd1ddf261f
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Sat, 2 Nov 2024 12:31:26 +0100
util.h: add a note about __dead and noreturn hints
- Add a note about the attribute for at least gcc/clang.
- _Noreturn has been deprecated.
- C23 introduces noreturn.
This is useful for testing with static analyzers like clang analyzer:
scan-build make
Running with scan-build make would return all false-positives for functions
using err(), errx() and die().
1c9433f80c0209c304236f84b58f87bd1ddf261f2024-11-01T15:59:14Z2024-11-01T15:59:14Zsfeedrc.5: fix typoHiltjo Posthumahiltjo@codemadness.orgcommit 1c9433f80c0209c304236f84b58f87bd1ddf261f
parent ec7abe8b5ea4c56bd442f536544c4dde4b38a847
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Fri, 1 Nov 2024 16:59:14 +0100
sfeedrc.5: fix typo
ec7abe8b5ea4c56bd442f536544c4dde4b38a8472024-10-27T08:23:53Z2024-11-01T15:52:45Zdocumentation improvementsHiltjo Posthumahiltjo@codemadness.orgcommit ec7abe8b5ea4c56bd442f536544c4dde4b38a847
parent 4b99778344bca3a1606046b5bf1087160ea1661c
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Sun, 27 Oct 2024 09:23:53 +0100
documentation improvements
- Add ETag example to sfeedrc.5 man page.
This was in the README but now also added to the man page for visibility.
- Indent code examples to separate leading text from code examples.
- Some textual improvements.
4b99778344bca3a1606046b5bf1087160ea1661c2024-09-09T14:43:28Z2024-09-09T14:43:28ZMakefile: on install do not follow symlinks and just copy themHiltjo Posthumahiltjo@codemadness.orgcommit 4b99778344bca3a1606046b5bf1087160ea1661c
parent 19957bc272e745af7b56b79fa648e8b6b77113b1
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Mon, 9 Sep 2024 16:43:28 +0200
Makefile: on install do not follow symlinks and just copy them
By default this is not needed, but it is used for my local script to bundle the
sfeed tools and compile/link and symlink them in a compact way (similar to
busybox).
The -R flag is needed on OpenBSD, because it ignores -P if the -R flag is not
specified. This issue is reported upstream, because POSIX defines -P can be
used without the -R flag.
19957bc272e745af7b56b79fa648e8b6b77113b12024-07-04T16:26:51Z2024-07-04T16:29:27Zsfeed.c: separator T could be lowercaseHiltjo Posthumahiltjo@codemadness.orgcommit 19957bc272e745af7b56b79fa648e8b6b77113b1
parent 1f6dfac83e8325a42dc5faa2e31cc25593acdfc8
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Thu, 4 Jul 2024 18:26:51 +0200
sfeed.c: separator T could be lowercase
From RFC3339:
5.6. Internet Date/Time Format
"
[...]
NOTE: Per [ABNF] and ISO8601, the "T" and "Z" characters in this
syntax may alternatively be lower case "t" or "z" respectively.
This date/time format may be used in some environments or contexts
that distinguish between the upper- and lower-case letters 'A'-'Z'
and 'a'-'z' (e.g. XML). Specifications that use this format in
such environments MAY further limit the date/time syntax so that
the letters 'T' and 'Z' used in the date/time syntax must always
be upper case. Applications that generate this format SHOULD use
upper case letters.
"
Reference: https://www.rfc-editor.org/rfc/rfc3339
1f6dfac83e8325a42dc5faa2e31cc25593acdfc82024-06-30T07:59:48Z2024-06-30T07:59:48Zimprove parsing whitespace after end tag namesHiltjo Posthumahiltjo@codemadness.orgcommit 1f6dfac83e8325a42dc5faa2e31cc25593acdfc8
parent 1f76c85ab88f0a1a36c7769a0b8ac13b9072105b
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Sun, 30 Jun 2024 09:59:48 +0200
improve parsing whitespace after end tag names
Simplified test-case:
https://git.codemadness.org/sfeed_tests/commit/e091160c3125322193bd8f27691c87eaa48cfc93.html
1f76c85ab88f0a1a36c7769a0b8ac13b9072105b2023-12-29T13:29:04Z2023-12-29T13:29:04Zbump version to 2.0Hiltjo Posthumahiltjo@codemadness.orgcommit 1f76c85ab88f0a1a36c7769a0b8ac13b9072105b
parent 391a556d308fe19b22614498d8bdefab0c3016be
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Fri, 29 Dec 2023 14:29:04 +0100
bump version to 2.0
391a556d308fe19b22614498d8bdefab0c3016be2023-12-29T13:21:30Z2023-12-29T13:21:30Zsfeed_update: reword some commentsHiltjo Posthumahiltjo@codemadness.orgcommit 391a556d308fe19b22614498d8bdefab0c3016be
parent 19cc32850dde2fe3fff1f02d8470727b84a5f67b
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Fri, 29 Dec 2023 14:21:30 +0100
sfeed_update: reword some comments
19cc32850dde2fe3fff1f02d8470727b84a5f67b2023-12-29T13:10:38Z2023-12-29T13:10:38Zsfeed_gopher: fix a clang-analyzer warningHiltjo Posthumahiltjo@codemadness.orgcommit 19cc32850dde2fe3fff1f02d8470727b84a5f67b
parent 04c76f86b66d2c30cadc6fef4df98cb3077ebcd9
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Fri, 29 Dec 2023 14:10:38 +0100
sfeed_gopher: fix a clang-analyzer warning
When reading from stdin the fields `total` and `totalnew` structure of feed are
incremented. However this value was uninitialized.
For stdin the result was not read and unused anyway though.
04c76f86b66d2c30cadc6fef4df98cb3077ebcd92023-12-29T12:56:42Z2023-12-29T12:56:42Zsfeed_update: code-style and consistency: add some comments for functionsHiltjo Posthumahiltjo@codemadness.orgcommit 04c76f86b66d2c30cadc6fef4df98cb3077ebcd9
parent cbf92f526528fc995e309d2f13b7dcebfd1e5c75
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Fri, 29 Dec 2023 13:56:42 +0100
sfeed_update: code-style and consistency: add some comments for functions
cbf92f526528fc995e309d2f13b7dcebfd1e5c752023-12-29T12:50:55Z2023-12-29T12:50:55Zsfeed_update/sfeed_opml_export: only allow regular filesHiltjo Posthumahiltjo@codemadness.orgcommit cbf92f526528fc995e309d2f13b7dcebfd1e5c75
parent 03a0a0ddb450f33e59a911a8be1b17d31507fa43
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Fri, 29 Dec 2023 13:50:55 +0100
sfeed_update/sfeed_opml_export: only allow regular files
Be more strict and only allow regular files. It makes no sense to use device
files or fifos with sfeed_update and it can cause issues, because sfeed_update
expects to read the config file for each (child) invocation also.
03a0a0ddb450f33e59a911a8be1b17d31507fa432023-12-28T15:48:06Z2023-12-28T15:48:06Zsfeed_opml_export: disallow using a directory as a config fileHiltjo Posthumahiltjo@codemadness.orgcommit 03a0a0ddb450f33e59a911a8be1b17d31507fa43
parent 0a5e36032373b34558e62f309be0b0ef7e925459
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Thu, 28 Dec 2023 16:48:06 +0100
sfeed_opml_export: disallow using a directory as a config file
Same as the commit 9754fe74f7b5c0600cc41eef8c6f5c8305a74a18 for sfeed_update
Tested on NetBSD 5.1: evaluating directories as config files could allow
garbage, so disallow it.
Devices / fifo, etc are still allowed.
0a5e36032373b34558e62f309be0b0ef7e9254592023-12-27T12:20:07Z2023-12-27T12:20:07Zsfeed_update: rename local variables just in caseHiltjo Posthumahiltjo@codemadness.orgcommit 0a5e36032373b34558e62f309be0b0ef7e925459
parent 1a5fa7454c92c5497c23e179e599224f0d96a920
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Wed, 27 Dec 2023 13:20:07 +0100
sfeed_update: rename local variables just in case
The config is loaded for each child program. These could override these
variables if the user specifies the same name.
1a5fa7454c92c5497c23e179e599224f0d96a9202023-12-27T12:18:03Z2023-12-27T12:18:03Zsfeed_update: remove xargs -sHiltjo Posthumahiltjo@codemadness.orgcommit 1a5fa7454c92c5497c23e179e599224f0d96a920
parent 8bdb849e449c5236a1ef7e8b4b91186790f7fb29
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Wed, 27 Dec 2023 13:18:03 +0100
sfeed_update: remove xargs -s
Theres no need to specify. POSIX defines it should support at least LINE_MAX
(2048 typically).
OpenIndiana xargs doesn't conform to POSIX. It doesn't use the largest constraint but
errors out.
From POSIX:
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/xargs.html
"Values of size up to at least {LINE_MAX} bytes shall be supported, provided
that the constraints specified in the DESCRIPTION are met. It shall not be
considered an error if a value larger than that supported by the implementation
or exceeding the constraints specified in the DESCRIPTION is given; xargs shall
use the largest value it supports within the constraints."
8bdb849e449c5236a1ef7e8b4b91186790f7fb292023-12-27T12:16:40Z2023-12-27T12:16:40Zsfeed_update: suppress output to stderr, like merge() already doesHiltjo Posthumahiltjo@codemadness.orgcommit 8bdb849e449c5236a1ef7e8b4b91186790f7fb29
parent 20bfe3d26d68d69539740110d3663b7853dad0e2
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Wed, 27 Dec 2023 13:16:40 +0100
sfeed_update: suppress output to stderr, like merge() already does
Noticed while testing TMPDIR=/noaccess sort on Illumos/OpenIndiana, which gives
a warning to stderr. For sort temporary directories might be used for large
output.
20bfe3d26d68d69539740110d3663b7853dad0e22023-12-26T15:38:14Z2023-12-26T18:07:30ZLICENSE: bump yearHiltjo Posthumahiltjo@codemadness.orgcommit 20bfe3d26d68d69539740110d3663b7853dad0e2
parent 87d5c99ebfee6e2255bd057e0eb45f1631b8b987
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Tue, 26 Dec 2023 16:38:14 +0100
LICENSE: bump year
87d5c99ebfee6e2255bd057e0eb45f1631b8b9872023-12-26T15:34:58Z2023-12-26T18:07:30Zsfeedrc: bump default maxjobs from 8 to 16Hiltjo Posthumahiltjo@codemadness.orgcommit 87d5c99ebfee6e2255bd057e0eb45f1631b8b987
parent 89dbf565de7b980d2b336c58633b7a27681adaf5
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Tue, 26 Dec 2023 16:34:58 +0100
sfeedrc: bump default maxjobs from 8 to 16
89dbf565de7b980d2b336c58633b7a27681adaf52023-12-26T15:37:21Z2023-12-26T18:07:30ZREADME: update dependency information for xargs for sfeed_updateHiltjo Posthumahiltjo@codemadness.orgcommit 89dbf565de7b980d2b336c58633b7a27681adaf5
parent cdb8f7feb135adf6f18e389b4bbf47886089474a
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Tue, 26 Dec 2023 16:37:21 +0100
README: update dependency information for xargs for sfeed_update
Remove the parallel xargs example because it is now the default.
cdb8f7feb135adf6f18e389b4bbf47886089474a2023-12-26T14:59:39Z2023-12-26T18:06:47Zsfeed_update: use xargs -P -0Hiltjo Posthumahiltjo@codemadness.orgcommit cdb8f7feb135adf6f18e389b4bbf47886089474a
parent 62bfed65ca91c34ea24b81b191c23d4542a7075b
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Tue, 26 Dec 2023 15:59:39 +0100
sfeed_update: use xargs -P -0
Some of the options, like -P are as of writing (2023) non-POSIX:
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/xargs.html. However
many systems support this useful extension for many years now.
Some historic context:
The xargs -0 option was added on 1996-06-11, about a year after the NetBSD
import (over 27 years ago at the time of writing):
http://cvsweb.openbsd.org/cgi-bin/cvsweb/src/usr.bin/xargs/xargs.c?rev=1.2&content-type=text/x-cvsweb-markup
On OpenBSD the xargs -P option was added on 2003-12-06 by syncing the FreeBSD
code:
http://cvsweb.openbsd.org/cgi-bin/cvsweb/src/usr.bin/xargs/xargs.c?rev=1.14&content-type=text/x-cvsweb-markup
Looking at the imported git history log of GNU findutils (which has xargs), the
very first commit already had the -0 and -P option on Sun Feb 4 20:35:16 1996
+0000.
Tested on many systems, old and new, some notable:
- OpenBSD 7.4
- Void Linux
- FreeBSD 12
- NetBSD 9.3
- HaikuOS (uses GNU tools).
- Slackware 11
- OpenBSD 3.8
- NetBSD 5.1
Some shells:
- oksh
- bash
- dash
- zsh
During testing there are some incompatibilities found in parsing the fields so
the arguments are passed as one argument which is split later on by the child
program.
62bfed65ca91c34ea24b81b191c23d4542a7075b2023-12-26T14:23:15Z2023-12-26T14:23:15Zsfeed_update: mktemp: improve compatibility with older systemsHiltjo Posthumahiltjo@codemadness.orgcommit 62bfed65ca91c34ea24b81b191c23d4542a7075b
parent 9754fe74f7b5c0600cc41eef8c6f5c8305a74a18
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Tue, 26 Dec 2023 15:23:15 +0100
sfeed_update: mktemp: improve compatibility with older systems
Tested on NetBSD 5.1:
- mktemp -p doesn't exist there yet.
- mktemp without any arguments/template doesnt work
- mktemp -d without any arguments/template doesnt work
9754fe74f7b5c0600cc41eef8c6f5c8305a74a182023-12-26T14:17:17Z2023-12-26T14:17:17Zsfeed_update: disallow using a directory as a config fileHiltjo Posthumahiltjo@codemadness.orgcommit 9754fe74f7b5c0600cc41eef8c6f5c8305a74a18
parent 7526bd914c314dd86cc7386bbfaf55ad0d130130
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Tue, 26 Dec 2023 15:17:17 +0100
sfeed_update: disallow using a directory as a config file
Tested on NetBSD 5.1: evaluating directories as config files could allow
garbage, so disallow it.
Devices / fifo, etc are still allowed.
7526bd914c314dd86cc7386bbfaf55ad0d1301302023-12-15T12:48:49Z2023-12-15T12:48:49ZREADME: sfeed_opml_export uses awkHiltjo Posthumahiltjo@codemadness.orgcommit 7526bd914c314dd86cc7386bbfaf55ad0d130130
parent a2aa09baf8a1f4a98313f8691d999eaff8b4ceea
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Fri, 15 Dec 2023 13:48:49 +0100
README: sfeed_opml_export uses awk
a2aa09baf8a1f4a98313f8691d999eaff8b4ceea2023-12-15T12:46:21Z2023-12-15T12:46:21Zsfeed_update: add die() function for exit and cleanup, respect $TMPDIRHiltjo Posthumahiltjo@codemadness.orgcommit a2aa09baf8a1f4a98313f8691d999eaff8b4ceea
parent 63308527f5197ddbcad6b06c5c1bbaf12f997e57
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Fri, 15 Dec 2023 13:46:21 +0100
sfeed_update: add die() function for exit and cleanup, respect $TMPDIR
- Add a die() helper function to cleanup and exit.
- NOTE that with an empty sfeedtmpdir the case rm -rf "" is fine.
- Respect $TMPDIR for creating temporary files like many UNIX tools do.
- Fix: when creating "${sfeedtmpdir}/ok" fails for some reason cleanup the
whole temporary directory as well.
- Fix: when the feeds() function is not defined exit with status code 1 (this
was incorrectly status code 0).
Reproduce: sfeed_update /dev/null; echo $?
63308527f5197ddbcad6b06c5c1bbaf12f997e572023-12-09T20:51:45Z2023-12-09T20:52:08Zimprove compatibility with zsh as a non-interactive shellHiltjo Posthumahiltjo@codemadness.orgcommit 63308527f5197ddbcad6b06c5c1bbaf12f997e57
parent da8adbc02e4667a68077e5f0cab8548e2bb81c1b
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Sat, 9 Dec 2023 21:51:45 +0100
improve compatibility with zsh as a non-interactive shell
In zsh the variables $path and $status are special.
https://zsh.sourceforge.io/Doc/Release/Parameters.html#index-path
https://zsh.sourceforge.io/Doc/Release/Parameters.html#index-status
(No promises I will keep up with this insanity in the future though)
da8adbc02e4667a68077e5f0cab8548e2bb81c1b2023-11-30T16:59:18Z2023-11-30T16:59:18Zsfeed_json: fix name of url field for attachmentsHiltjo Posthumahiltjo@codemadness.orgcommit da8adbc02e4667a68077e5f0cab8548e2bb81c1b
parent db69f0f81d4796816c04e6330b963f907cbfded1
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Thu, 30 Nov 2023 17:59:18 +0100
sfeed_json: fix name of url field for attachments
This was a typo: "url:" should be "url".
db69f0f81d4796816c04e6330b963f907cbfded12023-09-20T21:20:19Z2023-09-22T12:01:03Zsfeed_curses: mark functions as staticNRKnrk@disroot.orgcommit db69f0f81d4796816c04e6330b963f907cbfded1
parent a2af022b42beea7b8c29ca475f38efd10caf35e8
Author: NRK <nrk@disroot.org>
Date: Thu, 21 Sep 2023 03:20:19 +0600
sfeed_curses: mark functions as static
no reason for them not to be `static` and this also silences
-Wmissing-prototypes warning
(note: most of this patch was done by a sed(1) command.)
a2af022b42beea7b8c29ca475f38efd10caf35e82023-09-20T21:13:52Z2023-09-22T12:01:03Zsfeed_gopher: mark function as staticNRKnrk@disroot.orgcommit a2af022b42beea7b8c29ca475f38efd10caf35e8
parent c43e677d80f5eae1e05bcb71d88008fb8ded8cab
Author: NRK <nrk@disroot.org>
Date: Thu, 21 Sep 2023 03:13:52 +0600
sfeed_gopher: mark function as static
c43e677d80f5eae1e05bcb71d88008fb8ded8cab2023-08-18T16:40:05Z2023-08-18T16:40:05Zbump version to 1.9Hiltjo Posthumahiltjo@codemadness.orgcommit c43e677d80f5eae1e05bcb71d88008fb8ded8cab
parent 21a263cb27aeaf02b4a0a0319f435fac92f1ea28
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Fri, 18 Aug 2023 18:40:05 +0200
bump version to 1.9
21a263cb27aeaf02b4a0a0319f435fac92f1ea282023-08-15T17:10:51Z2023-08-15T17:14:16Zimprove wording and small typosHiltjo Posthumahiltjo@codemadness.orgcommit 21a263cb27aeaf02b4a0a0319f435fac92f1ea28
parent ea04ae0101ae0607295ebea0a23364d31d1d37aa
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Tue, 15 Aug 2023 19:10:51 +0200
improve wording and small typos
ea04ae0101ae0607295ebea0a23364d31d1d37aa2023-08-09T21:04:21Z2023-08-09T21:04:21ZMakefile: change -lcurses to -lncurses for Gentoo in the commentHiltjo Posthumahiltjo@codemadness.orgcommit ea04ae0101ae0607295ebea0a23364d31d1d37aa
parent 5e9dfb2a28a0305abb4e27df3b15214588c88468
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Wed, 9 Aug 2023 23:04:21 +0200
Makefile: change -lcurses to -lncurses for Gentoo in the comment
Reported by commodorian, thanks!
5e9dfb2a28a0305abb4e27df3b15214588c884682023-08-01T16:56:06Z2023-08-01T16:56:06Zsfeed_update.1: remove mentioning of / in path names hereHiltjo Posthumahiltjo@codemadness.orgcommit 5e9dfb2a28a0305abb4e27df3b15214588c88468
parent 3eb8fb2bb4fec18cc3e82b52086368bc837a41bd
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Tue, 1 Aug 2023 18:56:06 +0200
sfeed_update.1: remove mentioning of / in path names here
This is documented in the sfeedrc.5 also where it is more suitable.
3eb8fb2bb4fec18cc3e82b52086368bc837a41bd2023-08-01T16:55:49Z2023-08-01T16:55:49Zsfeed_curses.1: fix 2 typosHiltjo Posthumahiltjo@codemadness.orgcommit 3eb8fb2bb4fec18cc3e82b52086368bc837a41bd
parent d6eb193ec08f576d60cfe0e9f79e43fdb9c9634a
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Tue, 1 Aug 2023 18:55:49 +0200
sfeed_curses.1: fix 2 typos
d6eb193ec08f576d60cfe0e9f79e43fdb9c9634a2023-08-01T16:55:03Z2023-08-01T16:55:03Zsfeed_json.1: make the formatting consistent with other similar man pagesHiltjo Posthumahiltjo@codemadness.orgcommit d6eb193ec08f576d60cfe0e9f79e43fdb9c9634a
parent f2e7a3fca1c05599456a60494fff61bb7b1ecf4e
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Tue, 1 Aug 2023 18:55:03 +0200
sfeed_json.1: make the formatting consistent with other similar man pages
Such as sfeed_atom.1 or sfeed_twtxt.1
f2e7a3fca1c05599456a60494fff61bb7b1ecf4e2023-07-31T16:58:40Z2023-07-31T16:58:40Zfix typo: current -> currentlyHiltjo Posthumahiltjo@codemadness.orgcommit f2e7a3fca1c05599456a60494fff61bb7b1ecf4e
parent 92e833940f8a418157e84f8c1827c40a6a8a44c3
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Mon, 31 Jul 2023 18:58:40 +0200
fix typo: current -> currently
Reported by Allan Wind, thanks!
92e833940f8a418157e84f8c1827c40a6a8a44c32023-07-07T09:47:31Z2023-07-07T09:47:31Zsfeed_markread: fail early if creating a temporary file failedHiltjo Posthumahiltjo@codemadness.orgcommit 92e833940f8a418157e84f8c1827c40a6a8a44c3
parent c63b60816ff996c89a07131da411269152852cd7
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Fri, 7 Jul 2023 11:47:31 +0200
sfeed_markread: fail early if creating a temporary file failed
c63b60816ff996c89a07131da411269152852cd72023-07-07T09:43:41Z2023-07-07T09:43:41Zsfeedrc.5: document each feed name should be uniqueHiltjo Posthumahiltjo@codemadness.orgcommit c63b60816ff996c89a07131da411269152852cd7
parent 7afd4987e578631e41333867aa702e326cab87f1
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Fri, 7 Jul 2023 11:43:41 +0200
sfeedrc.5: document each feed name should be unique
Asked via e-mail. This confused a person which converted an OPML file which
yielded an sfeedrc with duplicate names or multiple "unnamed" entries.
This caused sfeed_update to fail, because it tried to merge files with the same
name and causing a race-condition because the temporary file with the same name
was already moved.
7afd4987e578631e41333867aa702e326cab87f12023-07-07T09:10:01Z2023-07-07T09:10:01Zsfeed_curses: move one line down when marking an item as read or unreadHiltjo Posthumahiltjo@codemadness.orgcommit 7afd4987e578631e41333867aa702e326cab87f1
parent 4601a1d688d05d300308d2bd861e16c1c81787a3
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Fri, 7 Jul 2023 11:10:01 +0200
sfeed_curses: move one line down when marking an item as read or unread
I don't mind either behaviour, but it has been suggested by a few people.
For example the mutt mail client also has this behaviour.
4601a1d688d05d300308d2bd861e16c1c81787a32023-07-01T10:51:04Z2023-07-02T10:06:36Zsfeedrc.5: remove branding (shaken, not stirred)Hiltjo Posthumahiltjo@codemadness.orgcommit 4601a1d688d05d300308d2bd861e16c1c81787a3
parent 229f33c4a66b8d4c60bfe39a068c34bf095f5728
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Sat, 1 Jul 2023 12:51:04 +0200
sfeedrc.5: remove branding (shaken, not stirred)
229f33c4a66b8d4c60bfe39a068c34bf095f57282023-06-09T12:45:57Z2023-06-09T12:45:57Zsfeedrc.5: improve example by setting a different User-AgentHiltjo Posthumahiltjo@codemadness.orgcommit 229f33c4a66b8d4c60bfe39a068c34bf095f5728
parent e96bdb644ae2f49125079e6218e2777caf633e1c
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Fri, 9 Jun 2023 14:45:57 +0200
sfeedrc.5: improve example by setting a different User-Agent
This is useful for sites that block clients that don't have a User-Agent header
set.
e96bdb644ae2f49125079e6218e2777caf633e1c2023-06-09T12:44:34Z2023-06-09T12:44:34ZREADME: improve note about CDNs and user-agent blockingHiltjo Posthumahiltjo@codemadness.orgcommit e96bdb644ae2f49125079e6218e2777caf633e1c
parent 44a4824eeba93a13511251a907179393f247c201
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Fri, 9 Jun 2023 14:44:34 +0200
README: improve note about CDNs and user-agent blocking
Prompted by a question from a user. Specificly mention reddit.com now blocks
clients which don't set the User-Agent header. It is specificly mentioned
because there is a reddit feed in the example sfeedrc.
44a4824eeba93a13511251a907179393f247c2012023-06-09T12:42:52Z2023-06-09T12:42:52Zadd comment to reference to the man pages and README fileHiltjo Posthumahiltjo@codemadness.orgcommit 44a4824eeba93a13511251a907179393f247c201
parent 96d7afc7d7511f05ba07c5acbb5bbfb2847bc126
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Fri, 9 Jun 2023 14:42:52 +0200
add comment to reference to the man pages and README file
These file contain examples and more details and may be overlooked/forgotten.
96d7afc7d7511f05ba07c5acbb5bbfb2847bc1262023-06-09T12:40:40Z2023-06-09T12:40:40Zsfeed_update/sfeedrc: add url as parameter to the filter() and order() functionHiltjo Posthumahiltjo@codemadness.orgcommit 96d7afc7d7511f05ba07c5acbb5bbfb2847bc126
parent 21790adad1672689225efe38b6a59494181d323b
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Fri, 9 Jun 2023 14:40:40 +0200
sfeed_update/sfeedrc: add url as parameter to the filter() and order() function
This might make it easier to set filters or ordering by pattern matching on a
group of feeds by the feed URL. For example youtube or reddit feeds.
Another way which was already possible is prefixing names with for example:
"reddit somename" or "yt somename".
21790adad1672689225efe38b6a59494181d323b2023-05-16T18:11:21Z2023-05-16T18:11:21Zimprove to use proper includesHiltjo Posthumahiltjo@codemadness.orgcommit 21790adad1672689225efe38b6a59494181d323b
parent 6245c2866bd38dcad51833f39677f99fb25fd0ef
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Tue, 16 May 2023 20:11:21 +0200
improve to use proper includes
Reduce using some of the unneeded sys/* headers too. This makes it slightly
more portable or easier to port also.
6245c2866bd38dcad51833f39677f99fb25fd0ef2023-05-16T16:30:09Z2023-05-16T16:30:09ZREADME: RSS 0.90+ is supported, not 0.91+Hiltjo Posthumahiltjo@codemadness.orgcommit 6245c2866bd38dcad51833f39677f99fb25fd0ef
parent bb89b1e4dbfc63857adb829c4fe31c63e9ca1b14
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Tue, 16 May 2023 18:30:09 +0200
README: RSS 0.90+ is supported, not 0.91+
See also:
https://www.rssboard.org/rss-0-9-0
bb89b1e4dbfc63857adb829c4fe31c63e9ca1b142023-05-15T16:53:19Z2023-05-15T16:53:19Zsfeed_atom: gmtime_r make it consistent with sfeed_mboxHiltjo Posthumahiltjo@codemadness.orgcommit bb89b1e4dbfc63857adb829c4fe31c63e9ca1b14
parent 5027669efb6a969afa2c8adcf7c2322dcf75de1f
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Mon, 15 May 2023 18:53:19 +0200
sfeed_atom: gmtime_r make it consistent with sfeed_mbox
When gmtime_r is called to get the current time show the same error message as
sfeed_mbox does.
5027669efb6a969afa2c8adcf7c2322dcf75de1f2023-05-15T16:48:56Z2023-05-15T16:48:56Zfix typoHiltjo Posthumahiltjo@codemadness.orgcommit 5027669efb6a969afa2c8adcf7c2322dcf75de1f
parent 3f7b8da537dca5b97657519eb8544f02f6b32d55
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Mon, 15 May 2023 18:48:56 +0200
fix typo
3f7b8da537dca5b97657519eb8544f02f6b32d552023-05-14T21:58:54Z2023-05-14T21:58:54Zsfeed_gopher: reduce scope and shadowing a variableHiltjo Posthumahiltjo@codemadness.orgcommit 3f7b8da537dca5b97657519eb8544f02f6b32d55
parent 538e7dc0ccf065bf59b952a46ff5658ea19f3bab
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Sun, 14 May 2023 23:58:54 +0200
sfeed_gopher: reduce scope and shadowing a variable
538e7dc0ccf065bf59b952a46ff5658ea19f3bab2023-05-14T21:38:17Z2023-05-14T21:38:17Zsfeed_curses.c: make struct urls static like the other variablesHiltjo Posthumahiltjo@codemadness.orgcommit 538e7dc0ccf065bf59b952a46ff5658ea19f3bab
parent 3909786d90499355617cba619d3668771b7a56d3
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Sun, 14 May 2023 23:38:17 +0200
sfeed_curses.c: make struct urls static like the other variables
3909786d90499355617cba619d3668771b7a56d32023-05-14T21:37:40Z2023-05-14T21:37:40Zxml.h: _XML_H_: macro name with an underscore is a reserved identifierHiltjo Posthumahiltjo@codemadness.orgcommit 3909786d90499355617cba619d3668771b7a56d3
parent bdcbf8589716c047a732db3cc349ee6114ccc25f
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Sun, 14 May 2023 23:37:40 +0200
xml.h: _XML_H_: macro name with an underscore is a reserved identifier
Found with clang -Wreserved-macro-identifier
See also:
https://wiki.sei.cmu.edu/confluence/display/c/DCL37-C.+Do+not+declare+or+define+a+reserved+identifier
bdcbf8589716c047a732db3cc349ee6114ccc25f2023-05-07T10:39:26Z2023-05-07T11:33:29Ziterate on previous commit which adds $SFEED_NEW_MAX_SECSHiltjo Posthumahiltjo@codemadness.orgcommit bdcbf8589716c047a732db3cc349ee6114ccc25f
parent ccee88cb02cb54f586d22c53e8bb8865817c3dc4
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Sun, 7 May 2023 12:39:26 +0200
iterate on previous commit which adds $SFEED_NEW_MAX_SECS
Separate the common pattern to get the time to compare new items in format
tools to the new util function: getcomparetime().
Some changes and notes:
- Change it so it is OK to set this value to 0 or negative (in the future).
- sfeed_curses: truncating newmaxsecs to an int would limit the value too much.
- Just use strtotime() and parse the value to time_t. This is a signed long
(32-bit, until 2038) or signed long long (64-bit) on most platforms.
- sfeed_curses gets the comparison time on reload aswell and it needs errno =
0, because it uses die(). time() is not guaranteed to set an errno if it fails.
- Rename environment variable to $SFEED_NEW_AGE.
ccee88cb02cb54f586d22c53e8bb8865817c3dc42023-05-07T10:56:46Z2023-05-07T11:29:56Zsfeed: datetounix: code-style, change , to separate lines (-Wcomma)Hiltjo Posthumahiltjo@codemadness.orgcommit ccee88cb02cb54f586d22c53e8bb8865817c3dc4
parent 6e4136753bd0faa15c198118b05c529d96d908c5
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Sun, 7 May 2023 12:56:46 +0200
sfeed: datetounix: code-style, change , to separate lines (-Wcomma)
6e4136753bd0faa15c198118b05c529d96d908c52023-04-24T20:01:23Z2023-05-07T11:22:55Zsfeed_{curses,frames,gopher,html,plain}: SFEED_NEW_MAX_SECSAlvar Penningpost@0x21.bizcommit 6e4136753bd0faa15c198118b05c529d96d908c5
parent aed47142611e23b363298bbb534c5cc4bd11a86b
Author: Alvar Penning <post@0x21.biz>
Date: Mon, 24 Apr 2023 22:01:23 +0200
sfeed_{curses,frames,gopher,html,plain}: SFEED_NEW_MAX_SECS
By introducing the new environment variable $SFEED_NEW_MAX_SECS in all
sfeed_* utilities marking feeds as new based on comparing their age, it
is now possible to override this age limit.
This allows, for example, to be notified about new feeds within the last
hour with
SFEED_NEW_MAX_SECS=3600 sfeed_plain ~/.sfeed/feeds/*
while creating a beautiful web report for last week's news by
SFEED_NEW_MAX_SECS=604800 sfeed_html ~/.sfeed/feeds/*
aed47142611e23b363298bbb534c5cc4bd11a86b2023-04-23T12:03:55Z2023-04-23T12:03:55Zbump version to 1.8Hiltjo Posthumahiltjo@codemadness.orgcommit aed47142611e23b363298bbb534c5cc4bd11a86b
parent eb5ca56e76013ac281129659efa2169ac125188e
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Sun, 23 Apr 2023 14:03:55 +0200
bump version to 1.8
eb5ca56e76013ac281129659efa2169ac125188e2023-04-14T17:11:38Z2023-04-14T17:11:38Zuse errno ENOMEM instead of EOVERFLOWHiltjo Posthumahiltjo@codemadness.orgcommit eb5ca56e76013ac281129659efa2169ac125188e
parent ebd7175e56141e8e8816b1b27f1abc773818b6aa
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Fri, 14 Apr 2023 19:11:38 +0200
use errno ENOMEM instead of EOVERFLOW
This matches the behaviour of setting errno for malloc/calloc on the following
systems too:
glibc, musl libc, OpenBSD libc.
It is also more portable for (older) systems.
ebd7175e56141e8e8816b1b27f1abc773818b6aa2023-04-14T17:11:03Z2023-04-14T17:11:03Zslightly reduce stack size for entitiesHiltjo Posthumahiltjo@codemadness.orgcommit ebd7175e56141e8e8816b1b27f1abc773818b6aa
parent eb8d6cf63815bff6697ebc7ae1b83f998b6eab53
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Fri, 14 Apr 2023 19:11:03 +0200
slightly reduce stack size for entities
A numeric entity could be 5 bytes, so use a round number of 8 bytes.
eb8d6cf63815bff6697ebc7ae1b83f998b6eab532023-04-12T22:34:23Z2023-04-12T22:34:23Zatom, json, mbox: fix reading past the buffer with an escaped NUL byte (\ NUL)Hiltjo Posthumahiltjo@codemadness.orgcommit eb8d6cf63815bff6697ebc7ae1b83f998b6eab53
parent 728270f69c34a84cb10aa891178c90c8fe36320d
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Thu, 13 Apr 2023 00:34:23 +0200
atom, json, mbox: fix reading past the buffer with an escaped NUL byte (\ NUL)
This would skip checking the end of the string of checking a NUL byte, because
the iteration was done before checking it.
It would proceed into the data that comes after. Note that sfeed itself can't
generate such malformed data itself.
Example input:
0 title link content\ html
Would incorrect print "contenthtml" as the content.
728270f69c34a84cb10aa891178c90c8fe36320d2023-04-12T20:55:54Z2023-04-12T20:55:54Zfix some typosHiltjo Posthumahiltjo@codemadness.orgcommit 728270f69c34a84cb10aa891178c90c8fe36320d
parent ced0dd7f8a01dedad3ba16c4cf209aea673c3e82
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Wed, 12 Apr 2023 22:55:54 +0200
fix some typos
ced0dd7f8a01dedad3ba16c4cf209aea673c3e822023-04-10T15:03:57Z2023-04-10T15:03:57Zsfeed_json: add JSON output format toolHiltjo Posthumahiltjo@codemadness.orgcommit ced0dd7f8a01dedad3ba16c4cf209aea673c3e82
parent 3f1d323497e5aefce65a3758d1bc408130d9bbe0
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Mon, 10 Apr 2023 17:03:57 +0200
sfeed_json: add JSON output format tool
This outputs the TSV data to JSON.
It uses a subset of JSON Feed 1.1:
https://www.jsonfeed.org/version/1.1/
3f1d323497e5aefce65a3758d1bc408130d9bbe02023-04-10T14:49:39Z2023-04-10T14:49:39Zsfeed.c: slightly reduce stack size for entitiesHiltjo Posthumahiltjo@codemadness.orgcommit 3f1d323497e5aefce65a3758d1bc408130d9bbe0
parent bf378f4350b0e92efcefd9c0c34f7b5c57c098f4
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Mon, 10 Apr 2023 16:49:39 +0200
sfeed.c: slightly reduce stack size for entities
A numeric entity could be 5 bytes, so use a round number of 8 bytes.
No other change intended and no performance difference noticed.
bf378f4350b0e92efcefd9c0c34f7b5c57c098f42023-04-10T14:44:13Z2023-04-10T14:44:13Zremove sfeed "branding" in outputsHiltjo Posthumahiltjo@codemadness.orgcommit bf378f4350b0e92efcefd9c0c34f7b5c57c098f4
parent 14cce37fa54608362dc2d19ff66d10b64e5777b1
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Mon, 10 Apr 2023 16:44:13 +0200
remove sfeed "branding" in outputs
Technically for sfeed_atom this violates the RFC standard. There must be an
author in the feed or for an item. However sfeed is not the author, it is a
generator. All readers will read these feeds anyway.
https://www.rfc-editor.org/rfc/rfc4287#section-4.1.1
"atom:feed elements MUST contain one or more atom:author elements, unless all of
the atom:feed element's child atom:entry elements contain at least one
atom:author element." *shrug*
14cce37fa54608362dc2d19ff66d10b64e5777b12023-04-10T14:40:56Z2023-04-10T14:40:56Zsfeed_atom: in Atom the default for a type is textHiltjo Posthumahiltjo@codemadness.orgcommit 14cce37fa54608362dc2d19ff66d10b64e5777b1
parent a15c98f32d8a768fee7b6783f7cea710cc5878a9
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Mon, 10 Apr 2023 16:40:56 +0200
sfeed_atom: in Atom the default for a type is text
Save a few bytes in the output by removing it.
https://www.rfc-editor.org/rfc/rfc4287
3.1.1. The "type" Attribute
The title is an atomTextConstruct.
a15c98f32d8a768fee7b6783f7cea710cc5878a92023-04-07T16:43:06Z2023-04-07T16:43:06Zsfeed.c: datetounix: add commentsHiltjo Posthumahiltjo@codemadness.orgcommit a15c98f32d8a768fee7b6783f7cea710cc5878a9
parent 0c938df27bf335fb290d6ea3612f5b397ed3da56
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Fri, 7 Apr 2023 18:43:06 +0200
sfeed.c: datetounix: add comments
0c938df27bf335fb290d6ea3612f5b397ed3da562023-04-06T18:22:54Z2023-04-06T18:22:54ZREADME: sfeed_download: change youtube-dl to yt-dlpHiltjo Posthumahiltjo@codemadness.orgcommit 0c938df27bf335fb290d6ea3612f5b397ed3da56
parent f11a83207e85e9007cf908a50318d50ef1e7bab9
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Thu, 6 Apr 2023 20:22:54 +0200
README: sfeed_download: change youtube-dl to yt-dlp
This is an active maintained fork.
f11a83207e85e9007cf908a50318d50ef1e7bab92023-04-06T18:22:00Z2023-04-06T18:22:00ZREADME: improve error checking for temporary files in examplesHiltjo Posthumahiltjo@codemadness.orgcommit f11a83207e85e9007cf908a50318d50ef1e7bab9
parent 15d18283616a951f060daf8f96996def532b8e21
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Thu, 6 Apr 2023 20:22:00 +0200
README: improve error checking for temporary files in examples
Similar to the recent fix for sfeed_update in commit
5a27c58675ddf4113d64a84f715cb3fecb681a6d
15d18283616a951f060daf8f96996def532b8e212023-04-06T17:35:24Z2023-04-06T17:35:24ZREADME: fix typo JSONfeed -> JSON FeedHiltjo Posthumahiltjo@codemadness.orgcommit 15d18283616a951f060daf8f96996def532b8e21
parent 5a27c58675ddf4113d64a84f715cb3fecb681a6d
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Thu, 6 Apr 2023 19:35:24 +0200
README: fix typo JSONfeed -> JSON Feed
https://www.jsonfeed.org/version/1.1/
5a27c58675ddf4113d64a84f715cb3fecb681a6d2023-04-05T19:41:19Z2023-04-05T19:41:19Zsfeed_update: fail early if creating a temporary directory or status file failsHiltjo Posthumahiltjo@codemadness.orgcommit 5a27c58675ddf4113d64a84f715cb3fecb681a6d
parent 92dfc055ce7a24bc8120c7eece3caca0ae84b653
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Wed, 5 Apr 2023 21:41:19 +0200
sfeed_update: fail early if creating a temporary directory or status file fails
If creating a temporary directory for the feed files failed then $sfeedtmpdir
would be empty and it would try to:
mkdir -p "/feed"
touch "/feed/ok"
After failing it would also still try to process all the feeds.
Now just fail early.
mktemp or touch themselve will print the actual error to stderr.
92dfc055ce7a24bc8120c7eece3caca0ae84b6532023-03-26T16:40:31Z2023-03-26T16:55:58Zsfeed.c: parsetime improve parsing RFC2822 obsolete short yearHiltjo Posthumahiltjo@codemadness.orgcommit 92dfc055ce7a24bc8120c7eece3caca0ae84b653
parent c7739b9c04b4b49888e1161164b1cdac0d9930ca
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Sun, 26 Mar 2023 18:40:31 +0200
sfeed.c: parsetime improve parsing RFC2822 obsolete short year
https://datatracker.ietf.org/doc/html/rfc2822#section-4.3
"Where a two or three digit year occurs in a date, the year is to be
interpreted as follows: If a two digit year is encountered whose
value is between 00 and 49, the year is interpreted by adding 2000,
ending up with a value between 2000 and 2049. If a two digit year is
encountered with a value between 50 and 99, or any three digit year
is encountered, the year is interpreted by adding 1900."
Improvement on commit 7086670e4335714e1df982bf1058082b7400b653
For example (output from TZ=UTC sfeed_plain):
input: Sun, 26 Jul 049 19:26:34
was: 2049-07-26 19:26
now: 1949-07-26 19:26 (because this is a 3-digit year)
input: Sun, 26 Jul 1 19:26:34
was: 2001-07-26 19:26
now: 0001-07-26 19:26 (because this is a 1-digit year and doesn't match the short year rule)
input: Sun, 26 Jul 001 19:26:34
was: 2001-07-26 19:26
now: 1901-07-26 19:26 (because this is a 3 digit year)
These cases are all added to the tests in the sfeed_tests repo (times.xml file).