json2tsv, branch HEAD

bump version to 1.2

2024-08-05T10:05:34Z

commit 66e7e402f861efe9d8853569e67a4bbaee877fc9 parent ac68c8941305cce69e5659123cffab04f6736d8a Author: Hiltjo Posthuma Date: Mon, 5 Aug 2024 12:05:34 +0200 bump version to 1.2

bump LICENSE year

2024-08-05T10:05:23Z

commit ac68c8941305cce69e5659123cffab04f6736d8a parent f689b5a0a309659de177c06bed733a0411f6ab02 Author: Hiltjo Posthuma Date: Mon, 5 Aug 2024 12:05:23 +0200 bump LICENSE year

fix: reset state for expecting key string in objects

2024-08-05T10:00:37Z

commit f689b5a0a309659de177c06bed733a0411f6ab02 parent 137d638b66af74f1547584a560ae7a90875b4bf1 Author: Hiltjo Posthuma Date: Mon, 5 Aug 2024 12:00:37 +0200 fix: reset state for expecting key string in objects Fixes an issue with the state machine still expecting a string to be an object key, even after an (empty) object has been parsed. This state is kept in the iskey variable, so reset it. This happens for example with empty objects {}. Test-case: {"test":[["a",{},"b"]]} Issue reported by Ben Collver, thanks!

bump version to 1.1

2023-05-25T20:42:04Z

commit 137d638b66af74f1547584a560ae7a90875b4bf1 parent 705f468584d0c425b6871a6e1f894e4c7aad0993 Author: Hiltjo Posthuma Date: Thu, 25 May 2023 22:42:04 +0200 bump version to 1.1

fix typo

2023-05-15T16:47:02Z

commit 705f468584d0c425b6871a6e1f894e4c7aad0993 parent 8ec8c8f26da6e20e58150a5b9e611ec651445469 Author: Hiltjo Posthuma Date: Mon, 15 May 2023 18:47:02 +0200 fix typo

json.h: _JSON_H_: macro name with an underscore is a reserved identifier

2023-05-14T21:32:19Z

commit 8ec8c8f26da6e20e58150a5b9e611ec651445469 parent 658cf342ce46348def6651d715b404a40baadb4f Author: Hiltjo Posthuma Date: Sun, 14 May 2023 23:32:19 +0200 json.h: _JSON_H_: macro name with an underscore is a reserved identifier Found with clang -Wreserved-macro-identifier See also: https://wiki.sei.cmu.edu/confluence/display/c/DCL37-C.+Do+not+declare+or+define+a+reserved+identifier

add a -u option: flush output after printing each value (unbuffered)

2023-04-23T10:39:38Z

commit 658cf342ce46348def6651d715b404a40baadb4f parent 994b5ebefdc81372822b42917e254ee1b0728db0 Author: Hiltjo Posthuma Date: Sun, 23 Apr 2023 12:39:38 +0200 add a -u option: flush output after printing each value (unbuffered) This works similar to the sed -u option. Sometimes it can be useful to force flushing the data directly. For example for streaming JSON data.

fail early if creating a temporary file failed

2023-04-23T10:22:51Z

commit 994b5ebefdc81372822b42917e254ee1b0728db0 parent 6bc2f0c4a81a8b4877385d396fac72e931a90e30 Author: Hiltjo Posthuma Date: Sun, 23 Apr 2023 12:22:51 +0200 fail early if creating a temporary file failed

json.c: use ENOMEM instead of EOVERFLOW

2023-04-14T17:13:12Z

commit 6bc2f0c4a81a8b4877385d396fac72e931a90e30 parent 3ee79e477e039f8028f1c47e253deeb8d293454a Author: Hiltjo Posthuma Date: Fri, 14 Apr 2023 19:13:12 +0200 json.c: use ENOMEM instead of EOVERFLOW

fix uninitialized variable (typo from previous commits)

2023-04-05T16:59:10Z

commit 3ee79e477e039f8028f1c47e253deeb8d293454a parent b58cef5c0b0297761836fd4ac0fd08f5a81f2eff Author: Hiltjo Posthuma Date: Wed, 5 Apr 2023 18:59:10 +0200 fix uninitialized variable (typo from previous commits)

bump LICENSE year

2023-04-04T16:34:06Z

commit b58cef5c0b0297761836fd4ac0fd08f5a81f2eff parent 403d7765995d21fbe24fda7318e8ac85c87bd4b1 Author: Hiltjo Posthuma Date: Tue, 4 Apr 2023 18:34:06 +0200 bump LICENSE year

json.c: add len argument to callback

2023-04-04T16:13:10Z

commit 403d7765995d21fbe24fda7318e8ac85c87bd4b1 parent 55e9552c536fcb18f28d6742cfbec3ae10766473 Author: Hiltjo Posthuma Date: Tue, 4 Apr 2023 18:13:10 +0200 json.c: add len argument to callback Not important for json2tsv itself, but for other uses of json.{c,h} it saves a strlen() call in some cases. Note that NUL bytes (via \u0000 or so) are discarded in json.c codepointtoutf8().

roll own ISDIGIT() macro instead of ctype isdigit()

2023-04-04T16:08:05Z

commit 55e9552c536fcb18f28d6742cfbec3ae10766473 parent 1f9c82d7086b221cb7a16cb41c3f870f8b08a552 Author: Hiltjo Posthuma Date: Tue, 4 Apr 2023 18:08:05 +0200 roll own ISDIGIT() macro instead of ctype isdigit() This ensures portable and consistent behaviour of checking a digit character.

bump version to 1.0

2022-09-01T17:13:24Z

commit 1f9c82d7086b221cb7a16cb41c3f870f8b08a552 parent fcf73ca5b42eb3ef32dc7d99d1b7bafa4a02df42 Author: Hiltjo Posthuma Date: Thu, 1 Sep 2022 19:13:24 +0200 bump version to 1.0

json2tsv.1: add simplified jaq example here too

2022-08-31T23:50:23Z

commit fcf73ca5b42eb3ef32dc7d99d1b7bafa4a02df42 parent c01b7afce194e126dd9d7db3965fe6209f052e76 Author: Hiltjo Posthuma Date: Thu, 1 Sep 2022 01:50:23 +0200 json2tsv.1: add simplified jaq example here too

jaq.1: small typo fix

2022-08-31T23:43:13Z

commit c01b7afce194e126dd9d7db3965fe6209f052e76 parent 7c73ed4db155770fb3c42edd396b894b209bdf41 Author: Hiltjo Posthuma Date: Thu, 1 Sep 2022 01:43:13 +0200 jaq.1: small typo fix

jaq: small tweaks and code-style

2022-08-31T23:39:35Z

commit 7c73ed4db155770fb3c42edd396b894b209bdf41 parent 6c4f4766a6151180fafe4f5d8d7e396e2a0d1876 Author: Hiltjo Posthuma Date: Thu, 1 Sep 2022 01:39:35 +0200 jaq: small tweaks and code-style

jaq: use jaq in the usage, not the full path which is ugly

2022-08-31T20:40:23Z

commit 6c4f4766a6151180fafe4f5d8d7e396e2a0d1876 parent 7a687c50f0dd18733249b3d6c553225ad85f9a3f Author: Hiltjo Posthuma Date: Wed, 31 Aug 2022 22:40:23 +0200 jaq: use jaq in the usage, not the full path which is ugly

json2tsv.1: reference jaq, not jtq

2022-08-31T20:11:56Z

commit 7a687c50f0dd18733249b3d6c553225ad85f9a3f parent faec67c1670631fcd8eaa7bfb9ac1f8f639b0621 Author: Hiltjo Posthuma Date: Wed, 31 Aug 2022 22:11:56 +0200 json2tsv.1: reference jaq, not jtq

jaq improvements

2022-08-31T20:07:59Z

commit faec67c1670631fcd8eaa7bfb9ac1f8f639b0621 parent 45d005ecfcbc9c6735e9ced2e9f9d31cd6cfe0ce Author: Hiltjo Posthuma Date: Wed, 31 Aug 2022 22:07:59 +0200 jaq improvements * simplify: make awk expressions mandatory and don't list the same format as json2tsv. If it is wanted just use json2tsv directly. * use $* (not the array $@) to assign to a string, noticed by shellcheck. * simplify: option parsing, allow using "-" as the start of an expression (except "-n"). * set an EXIT trap to really make sure the temporary file is removed. * improve documentation and add more details.

rename jtq to jaq

2022-08-30T22:21:47Z

commit 45d005ecfcbc9c6735e9ced2e9f9d31cd6cfe0ce parent 607615a6d80ffa57f81aca02f77ae94960828d63 Author: Hiltjo Posthuma Date: Wed, 31 Aug 2022 00:21:47 +0200 rename jtq to jaq

add jtq: a convenience wrapper script

2022-08-30T18:20:38Z

commit 607615a6d80ffa57f81aca02f77ae94960828d63 parent e2eeb2664324786e1a51e0ce64dbe03eeeff8346 Author: Hiltjo Posthuma Date: Tue, 30 Aug 2022 20:20:38 +0200 add jtq: a convenience wrapper script It wraps json2tsv, sets options for handling JSON data in a lossless manner and uses awk as a "query language".

json2tsv.1: properly escape backslashes

2022-05-02T18:16:47Z

commit e2eeb2664324786e1a51e0ce64dbe03eeeff8346 parent 2a02e40c0df8ea64f7cb59f15b867cdb1d9f6506 Author: Hiltjo Posthuma Date: Mon, 2 May 2022 20:16:47 +0200 json2tsv.1: properly escape backslashes From the mdoc(7) extended introduction: >To output a backslash, use the escape sequence \e. Never use the escape sequence \\ in any context. https://mandoc.bsd.lv/mdoc/intro/escaping.html (Thanks adc for reporting it to sfeed so it came to my attention).

bump version to 0.9

2022-04-02T16:31:38Z

commit 2a02e40c0df8ea64f7cb59f15b867cdb1d9f6506 parent 47732754f1217ac0808253b8c531428d82ec3a18 Author: Hiltjo Posthuma Date: Sat, 2 Apr 2022 18:31:38 +0200 bump version to 0.9

do not depend on C library locale for ctype functions/macros

2022-03-29T21:50:44Z

commit 47732754f1217ac0808253b8c531428d82ec3a18 parent 1ebc23030bb37e0f9399328ec3a6968e35758351 Author: Hiltjo Posthuma Date: Tue, 29 Mar 2022 23:50:44 +0200 do not depend on C library locale for ctype functions/macros

compatibility: replace iscntrl with own ISCNTRL macro

2022-03-28T16:42:11Z

commit 1ebc23030bb37e0f9399328ec3a6968e35758351 parent ca7aea61b3a593c782c7da48ec3e6552b47a698e Author: Hiltjo Posthuma Date: Mon, 28 Mar 2022 18:42:11 +0200 compatibility: replace iscntrl with own ISCNTRL macro It is unspecified if the C locale iscntrl is compatible with ASCII or not so force it to be.

improve stream handling of read and write errors

2022-03-19T12:10:53Z

commit ca7aea61b3a593c782c7da48ec3e6552b47a698e parent eb921bbad3f1120270014eaad9b2cba8a2958130 Author: Hiltjo Posthuma Date: Sat, 19 Mar 2022 13:10:53 +0100 improve stream handling of read and write errors

allow octal format and check the format a bit more strict

2022-02-21T16:16:38Z

commit eb921bbad3f1120270014eaad9b2cba8a2958130 parent f022807e57f81d93643e6126487e37f47132b13e Author: Hiltjo Posthuma Date: Mon, 21 Feb 2022 17:16:38 +0100 allow octal format and check the format a bit more strict

bump LICENSE year

2022-02-21T16:16:18Z

commit f022807e57f81d93643e6126487e37f47132b13e parent e889775a6c3962a1d043c5316beb956cd8f0cb05 Author: Hiltjo Posthuma Date: Mon, 21 Feb 2022 17:16:18 +0100 bump LICENSE year

json: fix code that could have a side-effects

2021-11-12T10:22:48Z

commit e889775a6c3962a1d043c5316beb956cd8f0cb05 parent 5c614ee9fef3c53e5c046b5fc2af19253a2899b5 Author: Hiltjo Posthuma Date: Fri, 12 Nov 2021 11:22:48 +0100 json: fix code that could have a side-effects It found a bug in the scc compiler.

README: optimize the unescape() function

2021-10-26T00:17:11Z

commit 5c614ee9fef3c53e5c046b5fc2af19253a2899b5 parent d0b6a90a0f4acb6b3635025c899553aad208f400 Author: Hiltjo Posthuma Date: Tue, 26 Oct 2021 02:17:11 +0200 README: optimize the unescape() function For bigger datasets use the -F and -R option though.

bump version to 0.8

2021-09-27T16:13:18Z

commit d0b6a90a0f4acb6b3635025c899553aad208f400 parent baf6b755ae6ade6117900996c134a86b12ade3b6 Author: Hiltjo Posthuma Date: Mon, 27 Sep 2021 18:13:18 +0200 bump version to 0.8

improve man page and documentation + fix a typo in a comment

2021-09-25T09:38:28Z

commit baf6b755ae6ade6117900996c134a86b12ade3b6 parent fea1a2013f259f23798d9a611f783f25b4f7448d Author: Hiltjo Posthuma Date: Sat, 25 Sep 2021 11:38:28 +0200 improve man page and documentation + fix a typo in a comment

print \n or \t also when using -F or -R

2021-09-25T09:37:20Z

commit fea1a2013f259f23798d9a611f783f25b4f7448d parent 65ee8e6903bc3c072a2e55fdd2501b0296fbdf59 Author: Hiltjo Posthuma Date: Sat, 25 Sep 2021 11:37:20 +0200 print \n or \t also when using -F or -R \n or \t is a control-character, but the intention of suppressing them by default is to not mess with the terminal (-r can be used to enable it).

use exitcode 3 for usage errors

2021-09-25T09:36:12Z

commit 65ee8e6903bc3c072a2e55fdd2501b0296fbdf59 parent 7c6507b10767029b3d4585e3e65d231f66fa904b Author: Hiltjo Posthuma Date: Sat, 25 Sep 2021 11:36:12 +0200 use exitcode 3 for usage errors

optimize printing the index of an array with the -n option

2021-09-24T12:27:38Z

commit 7c6507b10767029b3d4585e3e65d231f66fa904b parent 18215ba6f1a9f2c76ffceded0670eb2b2f466792 Author: Hiltjo Posthuma Date: Fri, 24 Sep 2021 14:27:38 +0200 optimize printing the index of an array with the -n option

add -F and -R option to allow to specify a different field and record separator

2021-09-18T14:17:32Z

commit 18215ba6f1a9f2c76ffceded0670eb2b2f466792 parent 226d85203f7ea26dcc71c98e0fb7fe3ffb78176b Author: Hiltjo Posthuma Date: Sat, 18 Sep 2021 16:17:32 +0200 add -F and -R option to allow to specify a different field and record separator

json: optimize number parsing for large number datasets

2021-09-23T16:31:00Z

commit 226d85203f7ea26dcc71c98e0fb7fe3ffb78176b parent 650481927d19d28d035d470bdedd2e2226e4dbd2 Author: Hiltjo Posthuma Date: Thu, 23 Sep 2021 18:31:00 +0200 json: optimize number parsing for large number datasets Measured on OpenBSD and Voidlinux glibc (gcc and clang). Not much difference on AMD Ryzen with GNUVoid Linux (glibc). Tested and much difference on OpenBSD-current (laptop, older Intel CPU), 6.8 and 6.9 (older Intel Xeons). Tested with json-testsuite, no differences (as intended).

bump version to 0.7

2021-04-29T16:14:56Z

commit 650481927d19d28d035d470bdedd2e2226e4dbd2 parent 770996c13b04532dacf3a190f57dd0c06ed4e7a7 Author: Hiltjo Posthuma Date: Thu, 29 Apr 2021 18:14:56 +0200 bump version to 0.7

separate JSON2TSV_* CFLAGS, LDFLAGS, use unlocked I/O by default

2021-04-29T10:57:45Z

commit 770996c13b04532dacf3a190f57dd0c06ed4e7a7 parent 917726e20fb722b0097e5be8a2f68ba385c58d66 Author: Hiltjo Posthuma Date: Thu, 29 Apr 2021 12:57:45 +0200 separate JSON2TSV_* CFLAGS, LDFLAGS, use unlocked I/O by default

json.c: fix a gcc warning

2021-04-29T10:56:56Z

commit 917726e20fb722b0097e5be8a2f68ba385c58d66 parent a6acda04e4f9f2285e126f7045b83bf7e2f8bf39 Author: Hiltjo Posthuma Date: Thu, 29 Apr 2021 12:56:56 +0200 json.c: fix a gcc warning gcc -Wall -o json.o -c json.c json.c: In function 'parsejson': json.c:94: warning: missing braces around initializer json.c:94: warning: (near initialization for 'nodes[0]')

README: add awk example function to unescape json2tsv output

2021-03-21T11:34:54Z

commit a6acda04e4f9f2285e126f7045b83bf7e2f8bf39 parent 84da1dac7041f1fc700cade27c55613a6f900de4 Author: Hiltjo Posthuma Date: Sun, 21 Mar 2021 12:34:54 +0100 README: add awk example function to unescape json2tsv output

LICENSE: bump year

2021-03-05T11:59:31Z

commit 84da1dac7041f1fc700cade27c55613a6f900de4 parent 88179a63c54bdc340781139fa91e3bd284bd6d73 Author: Hiltjo Posthuma Date: Fri, 5 Mar 2021 12:59:31 +0100 LICENSE: bump year

bump version to 0.6

2021-01-23T15:13:47Z

commit 88179a63c54bdc340781139fa91e3bd284bd6d73 parent b65bd5139faec35430d342dbce6c3b4bf802f4a8 Author: Hiltjo Posthuma Date: Sat, 23 Jan 2021 16:13:47 +0100 bump version to 0.6

json.c: fix utf-16 surrogate pair range

2021-01-21T23:23:19Z

commit b65bd5139faec35430d342dbce6c3b4bf802f4a8 parent 8128e7fa5d865ac7adff28e7ffb732f0b3b61f58 Author: Hiltjo Posthuma Date: Fri, 22 Jan 2021 00:23:19 +0100 json.c: fix utf-16 surrogate pair range Test-case of a high codepoint: U+10FFFD. Previously incorrect: printf '%s' '["\udbff\udffd"]' | json2tsv | hexdump -C 00000000 09 61 09 0a 5b 5d 09 73 09 ed af bf ed bf bd 0a |.a..[].s........| 00000010 Now correct: printf '%s' '["\udbff\udffd"]' | ./json2tsv | hexdump -C 00000000 09 61 09 0a 5b 5d 09 73 09 f4 8f bf bd 0a |.a..[].s......| 0000000e

json.h: add #ifdef guard

2020-11-04T16:55:40Z

commit 8128e7fa5d865ac7adff28e7ffb732f0b3b61f58 parent 58f7bb60f4e785121d30b808ce475407e4b95b5b Author: Hiltjo Posthuma Date: Wed, 4 Nov 2020 17:55:40 +0100 json.h: add #ifdef guard

Makefile: uninstall LICENSE, on install chmod $DOC files

2020-11-04T16:55:05Z

commit 58f7bb60f4e785121d30b808ce475407e4b95b5b parent 5b92fe10218579d7dd0c725c043de425d4076a9c Author: Hiltjo Posthuma Date: Wed, 4 Nov 2020 17:55:05 +0100 Makefile: uninstall LICENSE, on install chmod $DOC files Thanks adc!

Makefile: tiny compatibility improvement for tar -cf

2020-06-21T16:02:24Z

commit 5b92fe10218579d7dd0c725c043de425d4076a9c parent df5cda00ba8ef98787d15a666cd06b1fcb336b4c Author: Hiltjo Posthuma Date: Sun, 21 Jun 2020 18:02:24 +0200 Makefile: tiny compatibility improvement for tar -cf

Makefile: improvements

2020-06-10T12:38:45Z

commit df5cda00ba8ef98787d15a666cd06b1fcb336b4c parent 29d08288c8e16e76e408f2cab0729d7136b493fb Author: Hiltjo Posthuma Date: Wed, 10 Jun 2020 14:38:45 +0200 Makefile: improvements Pointed out by quinq, thanks * Use ar -rc (not ar rc) as defined in POSIX. * Install license as documentation too.

fix typo

2020-06-01T10:15:09Z

commit 29d08288c8e16e76e408f2cab0729d7136b493fb parent ee80731f6fa75e73aea2a30b4cf1f4bb939b6545 Author: Hiltjo Posthuma Date: Mon, 1 Jun 2020 12:15:09 +0200 fix typo

bump version to 0.5

2020-02-23T19:35:42Z

commit ee80731f6fa75e73aea2a30b4cf1f4bb939b6545 parent c91abbebb914af5ddac6f3d1c874a3b403172773 Author: Hiltjo Posthuma Date: Sun, 23 Feb 2020 20:35:42 +0100 bump version to 0.5

small white-space / code-style fix

2020-02-16T13:53:31Z

commit c91abbebb914af5ddac6f3d1c874a3b403172773 parent 2ba1ec15453197af2ef4fb5cf04157276fa70b67 Author: Hiltjo Posthuma Date: Sun, 16 Feb 2020 14:53:31 +0100 small white-space / code-style fix

cleanup some includes

2020-01-28T20:50:40Z

commit 2ba1ec15453197af2ef4fb5cf04157276fa70b67 parent 27d9d1e28a0d8fac91491ff2b0bfe964c081cba5 Author: Hiltjo Posthuma Date: Tue, 28 Jan 2020 21:50:40 +0100 cleanup some includes

update LICENSE year

2020-01-21T19:06:46Z

commit 27d9d1e28a0d8fac91491ff2b0bfe964c081cba5 parent f4ae5c240dbe7ca52eb9b7a7ab0a94a4fe3b6f06 Author: Hiltjo Posthuma Date: Tue, 21 Jan 2020 20:06:46 +0100 update LICENSE year

show usage on invalid flags or any arguments

2019-12-29T17:42:53Z

commit f4ae5c240dbe7ca52eb9b7a7ab0a94a4fe3b6f06 parent e569792c8bc41e19f1702ae57bbfbebc9fc64bff Author: Hiltjo Posthuma Date: Sun, 29 Dec 2019 18:42:53 +0100 show usage on invalid flags or any arguments

rename TYPE_* to JSON_TYPE_*

2019-12-29T17:34:39Z

commit e569792c8bc41e19f1702ae57bbfbebc9fc64bff parent cc245b50c91a1d3b3618773f289a3d4bb670df75 Author: Hiltjo Posthuma Date: Sun, 29 Dec 2019 18:34:39 +0100 rename TYPE_* to JSON_TYPE_*

bump version to 0.4

2019-11-12T00:22:56Z

commit cc245b50c91a1d3b3618773f289a3d4bb670df75 parent 13f4f439171cba26e0afaee4fc7eb98a483877c2 Author: Hiltjo Posthuma Date: Tue, 12 Nov 2019 01:22:56 +0100 bump version to 0.4

README: fix typo

2019-11-12T00:15:52Z

commit 13f4f439171cba26e0afaee4fc7eb98a483877c2 parent e0b9a599b21a1e85950af9f3185989d21c630970 Author: Hiltjo Posthuma Date: Tue, 12 Nov 2019 01:15:52 +0100 README: fix typo

update README

2019-11-09T15:19:45Z

commit e0b9a599b21a1e85950af9f3185989d21c630970 parent 33d5e77ed74ebe2ac9495ca3f4dbde493b26b69f Author: Hiltjo Posthuma Date: Sat, 9 Nov 2019 16:19:45 +0100 update README - remove -n flag information. This is not something for the introduction, but something for the manual and there is a -r flag now too. - structure sections similar to the sfeed README. - add list of build instructions. - add list of dependencies and optional dependencies section. - add author section.

add -r flag to allow printing all control-characters

2019-11-05T19:46:45Z

commit 33d5e77ed74ebe2ac9495ca3f4dbde493b26b69f parent a8950873f2e40fe767530b8e58b723c8b56abc33 Author: Hiltjo Posthuma Date: Tue, 5 Nov 2019 20:46:45 +0100 add -r flag to allow printing all control-characters No performance change with -O2 or higher with gcc or clang using no flags set. Minor tweaks: - rename showindices to nflag variable. - reword and improve formatting of man page. - improve parsing of options.

bump version to 0.3

2019-10-24T11:54:28Z

commit a8950873f2e40fe767530b8e58b723c8b56abc33 parent b780768500f3466bea0d2f770fc379eb9cb78669 Author: Hiltjo Posthuma Date: Thu, 24 Oct 2019 13:54:28 +0200 bump version to 0.3

optimize whitespace checking for gcc optimization

2019-10-23T19:39:49Z

commit b780768500f3466bea0d2f770fc379eb9cb78669 parent e49c809209014c1ab4e7c6b0921d82a5cbae5feb Author: Hiltjo Posthuma Date: Wed, 23 Oct 2019 21:39:49 +0200 optimize whitespace checking for gcc optimization Restores some performance. no difference for clang. gcc 4.2.1 clang 8.0.1

define all "expected" strings

2019-10-23T19:36:27Z

commit e49c809209014c1ab4e7c6b0921d82a5cbae5feb parent 29f995fe694b10e97eb61afa046de1915f0f97fb Author: Hiltjo Posthuma Date: Wed, 23 Oct 2019 21:36:27 +0200 define all "expected" strings

remove check for types and depth of object key string that cannot happen anymore

2019-10-23T17:40:36Z

commit 29f995fe694b10e97eb61afa046de1915f0f97fb parent b3cce42c19a31a92b232d9e6a2bdb973169a7878 Author: Hiltjo Posthuma Date: Wed, 23 Oct 2019 19:40:36 +0200 remove check for types and depth of object key string that cannot happen anymore

minor style changes, add some comments

2019-10-23T17:40:17Z

commit b3cce42c19a31a92b232d9e6a2bdb973169a7878 parent bb2e9c03c80acfe0a339c43b8d622713b14b8ea5 Author: Hiltjo Posthuma Date: Wed, 23 Oct 2019 19:40:17 +0200 minor style changes, add some comments

remove duplicate assignment of value, will be overwritten anyway

2019-10-23T17:34:59Z

commit bb2e9c03c80acfe0a339c43b8d622713b14b8ea5 parent 42251415a957f095453b290a96bcf870632eb58d Author: Hiltjo Posthuma Date: Wed, 23 Oct 2019 19:34:59 +0200 remove duplicate assignment of value, will be overwritten anyway

stylistic change: check EOF before strchr().

2019-10-23T17:07:17Z

commit 42251415a957f095453b290a96bcf870632eb58d parent 33c1ceffbcb04f8a6970d04841175f96737942bb Author: Hiltjo Posthuma Date: Wed, 23 Oct 2019 19:07:17 +0200 stylistic change: check EOF before strchr(). should have no functional difference, but check EOF first

fix regression: restore balanced nodes check for ,

2019-10-22T21:37:24Z

commit 33c1ceffbcb04f8a6970d04841175f96737942bb parent 69c080a4b190ea29c0dd1ae008c84cbd0712068f Author: Hiltjo Posthuma Date: Tue, 22 Oct 2019 23:37:24 +0200 fix regression: restore balanced nodes check for , The EXPECT_END state contained ',' which did not check the depth. reproduce: echo '"test",' | json2tsv This wrote nodes[-1].index++ and was treated as valid. a primitive followed by , was also valid (incorrect), for example: "true," EXPECT_END is "}]," All these states are checked now for the depth <= 0 so remove the EXPECT_NOTHING state.

json: declare symbols to JSON parsing internals static (local)

2019-10-20T14:46:21Z

commit 69c080a4b190ea29c0dd1ae008c84cbd0712068f parent 3877a663aff7b1a64e72717d5bb3f56b4ba46f99 Author: Hiltjo Posthuma Date: Sun, 20 Oct 2019 16:46:21 +0200 json: declare symbols to JSON parsing internals static (local)

json2tsv.1: document exit status

2019-10-20T13:01:22Z

commit 3877a663aff7b1a64e72717d5bb3f56b4ba46f99 parent 30f46f53a909c9d80b77ed3a5b2485d2f78ecc38 Author: Hiltjo Posthuma Date: Sun, 20 Oct 2019 15:01:22 +0200 json2tsv.1: document exit status

json2tsv.1: update man page documentation

2019-10-20T12:35:25Z

commit 30f46f53a909c9d80b77ed3a5b2485d2f78ecc38 parent 2f12ae56971af8acaa52357fe1042d37f71ffbd4 Author: Hiltjo Posthuma Date: Sun, 20 Oct 2019 14:35:25 +0200 json2tsv.1: update man page documentation

improvements

2019-10-20T12:30:24Z

commit 2f12ae56971af8acaa52357fe1042d37f71ffbd4 parent 9a85d6c5f69749ac71c690e5b4d36b8e70b15d16 Author: Hiltjo Posthuma Date: Sun, 20 Oct 2019 14:30:24 +0200 improvements - separate into more precise JSON types: primitives to: bool, null, number. - separate JSON code into a reusable "library": json.c. - remove errstr from parsejson(): just return an error code for out-of-memory and a JSON parse error. The tool returns exit code 2 when out-of-memory. - much more strict JSON parsing for incorrect input. - make primitives a fixed buffer. The longest size data for primitives are numbers and cannot be long anyway. - improve README example (reddit).

bump version to 0.2

2019-10-15T20:28:13Z

commit 9a85d6c5f69749ac71c690e5b4d36b8e70b15d16 parent 38f8dd3abd1ecbdf451ad9d98d707b09debe1c3a Author: Hiltjo Posthuma Date: Tue, 15 Oct 2019 22:28:13 +0200 bump version to 0.2

minor comment line change

2019-10-15T20:27:49Z

commit 38f8dd3abd1ecbdf451ad9d98d707b09debe1c3a parent d9837031a5640f6d55afd1cc4f370a54f1f6bdf0 Author: Hiltjo Posthuma Date: Tue, 15 Oct 2019 22:27:49 +0200 minor comment line change

man page: improve wording

2019-10-15T20:21:30Z

commit d9837031a5640f6d55afd1cc4f370a54f1f6bdf0 parent 4e539a2a08ad3348e8c4c2331b550e9ad24d69d9 Author: Hiltjo Posthuma Date: Tue, 15 Oct 2019 22:21:30 +0200 man page: improve wording

update/tweak comments about surrogate, shuffle one line

2019-10-15T20:14:19Z

commit 4e539a2a08ad3348e8c4c2331b550e9ad24d69d9 parent d8a06e34247a2219a62cb7b0d82218b87d155259 Author: Hiltjo Posthuma Date: Tue, 15 Oct 2019 22:14:19 +0200 update/tweak comments about surrogate, shuffle one line

remove this error type

2019-10-15T19:45:52Z

commit d8a06e34247a2219a62cb7b0d82218b87d155259 parent d9eb13696d70c0d87f13438cd9cb34d34102fb82 Author: Hiltjo Posthuma Date: Tue, 15 Oct 2019 21:45:52 +0200 remove this error type input: ] produces this error, which is unclear anyway.

bump max node depth from 32 to 64

2019-10-15T17:00:17Z

commit d9eb13696d70c0d87f13438cd9cb34d34102fb82 parent 9f4ab639718e4351b02b4bde6035cd588c32b169 Author: Hiltjo Posthuma Date: Tue, 15 Oct 2019 19:00:17 +0200 bump max node depth from 32 to 64 32 is already insanely high, but bump to 64 to be on the very safe side. I Compared this value to a few other JSON parsers.

handle unescaped string and unexpected EOF and improve handling surrogates

2019-10-15T16:58:56Z

commit 9f4ab639718e4351b02b4bde6035cd588c32b169 parent f0b7f8935d41162e29c5a01f15273ba225909969 Author: Hiltjo Posthuma Date: Tue, 15 Oct 2019 18:58:56 +0200 handle unescaped string and unexpected EOF and improve handling surrogates .... handle UTF-16 surrogate code-point errors and try to recover gracefully, just output the raw bytes.

separate error for invalid array members

2019-10-15T16:56:21Z

commit f0b7f8935d41162e29c5a01f15273ba225909969 parent 0329ac9b10b3d47b509afca6f82078bf50b32480 Author: Hiltjo Posthuma Date: Tue, 15 Oct 2019 18:56:21 +0200 separate error for invalid array members error: "member, but not in an object/array" example of such bad input: 1,2,3

treat unknown escape char as an error now

2019-10-15T16:55:24Z

commit 0329ac9b10b3d47b509afca6f82078bf50b32480 parent d58761df609fb30dce860560b849f7f2d389e007 Author: Hiltjo Posthuma Date: Tue, 15 Oct 2019 18:55:24 +0200 treat unknown escape char as an error now

allow primitive and string type on its own (without object or array)

2019-10-15T16:53:51Z

commit d58761df609fb30dce860560b849f7f2d389e007 parent 38d0984744efb248bfe773431d922a63cd09e0e3 Author: Hiltjo Posthuma Date: Tue, 15 Oct 2019 18:53:51 +0200 allow primitive and string type on its own (without object or array) now valid: 1 true null "string"

make wrong parsing a wrong string a bit more logical

2019-10-15T16:50:21Z

commit 38d0984744efb248bfe773431d922a63cd09e0e3 parent 216c4f342f22413073cfa9a98cb069b93eb0ecb4 Author: Hiltjo Posthuma Date: Tue, 15 Oct 2019 18:50:21 +0200 make wrong parsing a wrong string a bit more logical test: {"a" "b"} result: handle as "ab" instead of ignoring "a" {1"a"} is now: "1a" (string) instead of "a"

allow 0x7f (DEL) inside strings (allowed in the RFC)

2019-10-15T16:49:17Z

commit 216c4f342f22413073cfa9a98cb069b93eb0ecb4 parent 03e5cd467dd7219175dc25fe821cea5e6d2a5520 Author: Hiltjo Posthuma Date: Tue, 15 Oct 2019 18:49:17 +0200 allow 0x7f (DEL) inside strings (allowed in the RFC)

define errors are const, this allows easy comparison too

2019-10-15T16:48:05Z

commit 03e5cd467dd7219175dc25fe821cea5e6d2a5520 parent f8704af3ceec42fb7be10caca4392c85f82631eb Author: Hiltjo Posthuma Date: Tue, 15 Oct 2019 18:48:05 +0200 define errors are const, this allows easy comparison too

different scaling for memory allocation

2019-10-15T16:43:56Z

commit f8704af3ceec42fb7be10caca4392c85f82631eb parent dfd26e7ccf4f83f4c140503b69d773b1dbb81808 Author: Hiltjo Posthuma Date: Tue, 15 Oct 2019 18:43:56 +0200 different scaling for memory allocation greedier for large strings, much less memory for smaller ones

check a capacity before a write

2019-10-14T22:48:48Z

commit dfd26e7ccf4f83f4c140503b69d773b1dbb81808 parent c12ff5a298eaa6efaa2651b5ea16dde24dae1147 Author: Hiltjo Posthuma Date: Tue, 15 Oct 2019 00:48:48 +0200 check a capacity before a write

check capacity before each write, reset name of object keys always on ':'

2019-10-14T22:46:31Z

commit c12ff5a298eaa6efaa2651b5ea16dde24dae1147 parent 65cdb7d2d32ee17246328b50c5a0ae4deab585fb Author: Hiltjo Posthuma Date: Tue, 15 Oct 2019 00:46:31 +0200 check capacity before each write, reset name of object keys always on ':'

fix for empty key in objects

2019-10-14T22:19:24Z

commit 65cdb7d2d32ee17246328b50c5a0ae4deab585fb parent 794f0a187d7ed4d72e60fbad7186e17f71ca23c4 Author: Hiltjo Posthuma Date: Tue, 15 Oct 2019 00:19:24 +0200 fix for empty key in objects reproduce: {"a":1,"":"b"}

handle empty strings correctly

2019-10-14T21:08:23Z

commit 794f0a187d7ed4d72e60fbad7186e17f71ca23c4 parent 78d6bc291c601ffda102b3572c642bd79f415449 Author: Hiltjo Posthuma Date: Mon, 14 Oct 2019 23:08:23 +0200 handle empty strings correctly reproduce: [""]

check unclosed arrays/objects more strictly

2019-10-14T20:55:03Z

commit 78d6bc291c601ffda102b3572c642bd79f415449 parent d45f1447a26996cfbaf80c5a37a3ba51a48aecb8 Author: Hiltjo Posthuma Date: Mon, 14 Oct 2019 22:55:03 +0200 check unclosed arrays/objects more strictly

check closing of object and array in a more logical and strict way

2019-10-14T20:53:19Z

commit d45f1447a26996cfbaf80c5a37a3ba51a48aecb8 parent 1232e02c50535bb4f9b363a2dada9525260ed059 Author: Hiltjo Posthuma Date: Mon, 14 Oct 2019 22:53:19 +0200 check closing of object and array in a more logical and strict way (should have no functional difference though, since it is unreachable)

simplify type printing

2019-10-14T20:51:15Z

commit 1232e02c50535bb4f9b363a2dada9525260ed059 parent 92f3832c6fb144de7a13d962327128368912907b Author: Hiltjo Posthuma Date: Mon, 14 Oct 2019 22:51:15 +0200 simplify type printing

be much less memory greedy

2019-10-14T20:49:46Z

commit 92f3832c6fb144de7a13d962327128368912907b parent 058bab370580f79697a492b58a8547a5deae6e64 Author: Hiltjo Posthuma Date: Mon, 14 Oct 2019 22:49:46 +0200 be much less memory greedy

simplify calculation: the low surrogate range is already checked

2019-10-13T19:40:11Z

commit 058bab370580f79697a492b58a8547a5deae6e64 parent 933582372d81a911193fb1da7c86b6b960432535 Author: Hiltjo Posthuma Date: Sun, 13 Oct 2019 21:40:11 +0200 simplify calculation: the low surrogate range is already checked

add support for high-low surrogates and UTF-16 decoding

2019-10-13T19:31:31Z

commit 933582372d81a911193fb1da7c86b6b960432535 parent 922491e0343ab6f440024803921daf843b0e9cf5 Author: Hiltjo Posthuma Date: Sun, 13 Oct 2019 21:31:31 +0200 add support for high-low surrogates and UTF-16 decoding seen in the wild on a Reddit JSON file encoding emojis. It is also mentioned in the RFC7159 - 7. Strings

documentation: use the name TAB-Separate Value format (TSV)

2019-10-13T12:52:39Z

commit 922491e0343ab6f440024803921daf843b0e9cf5 parent 30c28c0ba7a3bec23f3c964f59aef85a9c2ba615 Author: Hiltjo Posthuma Date: Sun, 13 Oct 2019 14:52:39 +0200 documentation: use the name TAB-Separate Value format (TSV)

fix integer overflow check

2019-10-13T12:51:50Z

commit 30c28c0ba7a3bec23f3c964f59aef85a9c2ba615 parent 79e73bfa51005f9b760c01117f4decce4435fe29 Author: Hiltjo Posthuma Date: Sun, 13 Oct 2019 14:51:50 +0200 fix integer overflow check

improve documentation, reference man page in the README

2019-10-06T14:31:37Z

commit 79e73bfa51005f9b760c01117f4decce4435fe29 parent 60cdcd6ec005e72e168f0318228371471121ad9b Author: Hiltjo Posthuma Date: Sun, 6 Oct 2019 16:31:37 +0200 improve documentation, reference man page in the README

optimize and simplify a bit: allow more efficient buffering

2019-10-06T14:20:13Z

commit 60cdcd6ec005e72e168f0318228371471121ad9b parent f831a4b76c10d7efcb72effd18332f3deac7a4fa Author: Hiltjo Posthuma Date: Sun, 6 Oct 2019 16:20:13 +0200 optimize and simplify a bit: allow more efficient buffering on a sample input using getchar_unlocked and putchar_unlocked parsed 113MB of JSON data in 1.7 seconds vs 2.7. Tested with gcc and clang.