Enhancement: Write the bug-report.sh tool

Add a make report that would run the test suite and write various info to say bug-report.txt

The info would include version numbers, out of commands such as uname -a, compiler version info, contents of limit_ioccc.sh, encourage the user to report the contents of the bug-report.txt file in a GitHub bug report, etc.

Help wanted: compressed tarball filename sanity check tool: filenamechk

Discussed in https://github.com/ioccc-src/mkiocccentry/discussions/39

^{Originally posted by lcn2 January 29, 2022}

We need help with more tools in this repo

You can help. See below.

We are working hard to be able to run a MOCK IOCCC in 2022

By MOCK IOCCC we mean a beta test run of an IOCCC where entries submitted would NOT be judged. Instead the IOCCC judges would go though the motions of running mock contest in order to beta test the process. Nobody would win the MOCK IOCCC, although we would certainly thank those who helped test the process with special thanks to those who found something in the MOCK process that we need to fix.

Any important lessons learned during the MOCK IOCCC would result in changes to the real IOCCC28.

Our goal is to hold the MOCK IOCCC in 2022 and then start the reason IOCCC28 in 2022.

There is A LOT of work that goes on behind the scenes by the IOCCC judges. You can help the IOCCC judges by helping us with the following tools related to mkiocccentry. See below.

JSON sanity check tools

Write a tool that attempts validate (performs sanity checks) on the .info.json produced by mkiocccentry.
Write a tool that attempts validate (performs sanity checks) on the .author.json produced by mkiocccentry

Those two tools could be written as pull requests to add them as separate programs to this repo. Moreover mkiocccentry could be modified to perform a function call into that code (in the same way the code iocccsize function is executed directly from mkiocccentry while iocccsize is also compiled as a separate tool).

Even though mkiocccenty would run the core of those tools as function calls, we still need the tools as standalone programs for use elsewhere. So both mkiocccentry callable functions AND standalone tools are needed.

The standalone tools should take a file as an argument. If sane, these tools should exit 0, otherwise exit non-zero.

compressed tarball filename sanity check tool

Write a tool that validates the filename of a compressed tarball file.

The mkiocccenty tool forms a compressed tarball file with a very specific form of a filename (for internal IOCCC judging process reasons). We need a tool that performs a sanity check on the filename of a compressed tarball file.

That tool could be added to this repo as a pull request. Moreover mkiocccentry could be modified to perform a function call into that code (in the same way the code iocccsize function is executed directly from mkiocccentry while iocccsize is also compiled as a separate tool) as a sanity check of what mkiocccentry produced.

Even though mkiocccenty would run the core of those tools as function calls, we still need the tool as standalone programs for use elsewhere. So both mkiocccentry callable functions AND standalone tool are needed.

The standalone tool should take a file as an argument. If sane, these tool should exit 0, otherwise exit non-zero.

compressed tarball sanity check tool

Write a tool to validate a compressed tarball produced by mkiocccentry

Such a compressed tarball tool would need to perform safety and sanity checks on the compressed tarball WITHOUT uncompressing it on disk. It would need to be paranoid, expecting the worst while attempting to validate for the best.

For example, it could run "tar -t" and look at the output to verify that the files un-tar into a sub-directory (i.e., not wander off with ../../../foo path, not attempting to write something under /foo).

It would also call the code of the "validate the filename of the compressed tarball file" (in the same way the code iocccsize function is executed directly from mkiocccentry while iocccsize is also compiled as a separate tool).

This tool would need to verify that the sub-directory that the compressed tarball would produce is related to the previously validated compressed tarball filename.

This compressed tarball tool would need to perform the two checks on the tarball size both checking the size of the compressed tarball and tar reports as the overall size of the elements of the tarball (see limit_ioccc.h).

This sanity check tool should verify that the compressed tarball has the required files (prog.c, Makefile, remarks.md, .info.json, .author.json).

The sanity check tool should be sure that no other files beginning with "." would be created. It should check that no sub-sub-directories would be created, just the entry directory from which the tarball was created in the first place. It should check that all files created are UNDER the entry directory only.

That tool could be added to this repo as a pull request. Moreover mkiocccentry could be modified to perform a function call into that code (in the same way the code iocccsize function is executed directly from mkiocccentry while iocccsize is also compiled as a separate tool) as a sanity check of what mkiocccentry produced.

Even though mkiocccenty would run the core of those tools as function calls, we still need the tool as standalone programs for use elsewhere. So both mkiocccentry callable functions AND standalone tool are needed.

The standalone compress tarball sanity check tool should take a file as an argument. If sane, these tool should exit 0, otherwise exit non-zero.

In general

The reason why mkiocccentry should be able to execute the sanity check code of the above mention tools is that we (the IOCCC judges) plan to use those standalone tools in our judging procedures. Nevertheless mkiocccentry should perform the same sanity checks to be help be sure what is generates will later will be checked by the IOCCC judges via the standalone tools.

We don't want a mismatch of what mkiocccentry produces to conflict with the result of the sanity check standalone tools. We don't want someone's IOCCC entry they worked hard on, invalidated because mkiocccenty produced something that the standalone tool later rejected. That is why mkiocccentry should call on the sanity check code to validate what it is doing / creating.

Instead of loading up mkiocccentry with fork/exec of calling standalone tools and parsing the results, also allow function calls from mkiocccentry to run the same sanity checking code.

These sanity check tools need to be written in a clean, highly readable, trivial to understand style. Ironic with such a request given that this is in support of Obfuscated C, yes, but IOCCC is about irony in a way.

We want it to be easy for us and others to inspect what this code is doing. We want it easy for this code to be modified and improved on in the future.

We want this code to stand as a good example of how to write well written, easy to understand, easy to improve C code.

Use of the dbg facility, and the -v level debugging style used by mkiocccentry will be important. And if a similar coding style with ALL libc function calls checked for errors is also important.

And as a C contest, we would highly prefer these tools be written in portable C and compiled with gcc or clang without warnings.

In summary

Your help wanted AND is appreciated!

Enhancement: write the hostchk.sh code

The make hostchk rule performs some basic checks to see if the host can support this repo.

The actions of make hostchk should be does via a hostchk.sh shell script.

We do NOT need to test everything about the host, just a few basic elements.

For example, test if tar as the required flags such as -J and --format=v7.

If a problem is detected, a suggestion could be given to the user. For example, if they are running under an OS that does not have a modern enough tar (such as perhaps IRIX or maybe an old OpenBSD), one should recommend they download a install a more up to date tar such as GNU Tar.

implicit-fallthrough warning

$ make
cc -O3 -g3 --pedantic -Wall -Wextra mkiocccentry.c -o mkiocccentry
mkiocccentry.c: In function ‘main’:
mkiocccentry.c:827:6: warning: this statement may fall through [-Wimplicit-fallthrough=]
      usage(0, __func__, "-h help mode:\n", program, tar, cp, ls);
      ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
mkiocccentry.c:829:2: note: here
  case 'v': /* -v verbosity */
  ^~~~

Inserting break fixes this:

 	case 'h':		/* -h - print help to stderr and exit 0 */
 	    usage(0, __func__, "-h help mode:\n", program, tar, cp, ls);
 	    /*NOTREACHED*/
+	    break;
 	case 'v':	/* -v verbosity */

Enhancement: Discuss the debug facility documentation and code so it can be split to another repo

Rather than clutter up issue #171 I think we can discuss the concerns here. As I want to go afk again (in fact need to go afk again) I will not copy over the comments regarding it. If you've not replied to any and I have the time (and not working on the actual changes) maybe I will copy them over here.

Going afk a while ... not sure when I'll be back. Feel free to assign this issue to me - and to you if you wish.

Enhancement: limit use of Centos 7 -D flags

Background

With commit f85ec5a we uncovered a significant problem with how C files are compiled while trying to also support old platforms such as Centos 7:

There is a significant problem with using:

    -D_DEFAULT_SOURCE -D_ISOC99_SOURCE -D_POSIX_C_SOURCE=200809L -D_XOPEN_SOURCE=600

These defines prevent functions that were introduced into C post c99
from being compiled without a warning/error. Take for example, the
use of timegm(3) in entry_util.c:

    cc -D_DEFAULT_SOURCE -D_ISOC99_SOURCE -D_POSIX_C_SOURCE=200809L -D_XOPEN_SOURCE=600 -std=gnu11 -O0 -g -pedantic -Wall -Wextra -Werror -Wno-unused-command-line-argument -Wno-poison-system-directories -Wno-unreachable-code-break -Wno-padded -lm entry_util.c -c

will produce the following error:

    entry_util.c:1150:25: error: implicit declaration of function 'timegm' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
        timestr_as_time_t = timegm(&timeptr);

Whereas this:

    cc -std=gnu11 -O0 -g -pedantic -Wall -Wextra -Werror -Wno-unused-command-line-argument -Wno-poison-system-directories -Wno-unreachable-code-break -Wno-padded -lm entry_util.c -c

will compile successfully.

While we understand that certain older systems such as CentOS 7 need:

    -D_DEFAULT_SOURCE -D_ISOC99_SOURCE -D_POSIX_C_SOURCE=200809L -D_XOPEN_SOURCE=600

some of the time, other systems CANNOT use them as they force the compiler,
in certain cases, to process system header files as of -std=c99 were in effect
even though -std=gnu11 is being used.

Even where modern systems do not generate warnibngs/errors, the use
of those defines could have adverse impacts in system include files
that might be incompatible with the C standard library being invoked
by the -std flag.

Makefile changes

To address this somewhat, the following changes to Makefile were made:

The ${STD_SRC} has been changed to:

D_LEGACY= -D_DEFAULT_SOURCE -D_ISOC99_SOURCE -D_POSIX_C_SOURCE=200809L -D_XOPEN_SOURCE=600
C_STD= -std=gnu11

The ${CFLAGS} is now:

CFLAGS= ${C_STD} ${COPT} -pedantic ${WARN_FLAGS} ${LDFLAGS}

A new ${LEGACY_CFLAGS} was added:

LEGACY_CFLAGS= ${D_LEGACY} ${CFLAGS

To compile this change, all compile lines with ${CC} use ${LEGACY_CFLAGS}
except for entry_util.o which uses ${CFLAGS}.

Task needed

On a Centos 7 platform, convert as many uses of ${LEGACY_CFLAGS}
in the Makefile to just ${LEGACY_CFLAGS} and still compile the repo.

Of if this is NOT possible, say because entry_util.c cannot be compiled
on Centos 7 with just ${CFLAGS}, then choose to abandon the
Centos 7 platform as a supported platform for this repo.

NOTE: If Centos 7 were to be abandoned, people could still submit
IOCCC entries developed on Centos 7. They would just have to
find a different platform to compile this repo to form their IOCCC compressed tarball.

Question: Should the mkiocccentry tool help the user by deleting the temporary directory used to form the tarball ?

It seems like it might be a nice feature to add since it's possible for many that they update their entry (entries). They could do it manually and they would have to if the tool failed but I think this might make the tool just a bit easier to use, removing the burden from the user.

I'm happy to do this as it would be useful to me and probably others but if this is not desirable I'd rather not take the time to do this. What are the judges thoughts here?

No new forks allowed?

See screenshot. Not sure if this was a mistake or something else going on but:

Enhancement: finish the C-based general JSON parser

As I said in the other thread I am also just typing on my phone and I am about to go get some sleep but I can answer your questions tomorrow @lcn2.

As I also said I will be gone most of Saturday and probably will take a few days to recover.

Finally feel free to edit the title and this message or else let me know what you think is a good start.

We can then discuss what I have so far and how we should proceed with the parser.

I will say quickly before I go that I have kind of held back a bit to see where the structs (or the functions acting on them) go as I think it will be helpful to have something more to work on. Once the structs are sorted it should be easier to actually write the rules and actions in the lexer and parser.

I am not sure but there very possibly is missing grammar too.

Hope this is a good start for the issue. Have a good rest of your day and look forward seeing more here my friend!

Have a safe trip home when you go back.

TODO:

Make better error messages for invalid input (and all errors in general).

Broken indentation around get_title().

There's a broken indentation around get_title():

    /*
     * get_title - get the title of the entry
     *
     * Ask the user for an entry title, validate the response
     * and return the malloced title.
     *
     * given:
     *      infop   - pointer to info structure
     *
     * returns:
     *      malloced and validated title
     *
     * This function does not return on error.
     */
    static char *
    get_title(struct info *infop)
    {
	char *title = NULL;		/* entry title to return or NULL */
	size_t len;			/* length of title */
	size_t span;		/* span of valid characters in title */
	int ret;			/* libc function return */

After that "else if" the wrong indentation starts.

	} else if (strcmp(response, "n") == 0 || strcmp(response, "no") == 0) {

		/*
		 * free storage
		 */

The wrong indentation ends here at "title = prompt(...":

	    /*
	     * obtain the reply
	     */
	    title = prompt("Enter a title for your entry", NULL);

	/*
	 * title cannot be empty
	 */

Pre-formal release tracker

I thought of this earlier today and I just wrote something that made me think of it again at the mostly OT thread. The comment #275 (comment) includes the text:

There is one benefit of not having the man pages (+ other documentation?) for the debug and dynamic array facilities in this repo that I just thought of:

It would mean there are fewer things that have to be done before IOCCCMOCK can happen. We could discuss the documentation in their respective repos (well once the dynamic array one is created .. unless you think we should open an issue here for it as a kind of workaround for the fact the repo does not exist yet?) on the side but if we don't focus on them now it would mean only the JSON semantics tables tools would need to be documented and then the finishing of chkentry + go through everything for typos / consistencies etc.

What do you think of that?

Speaking of such: do we need the JSON parser to be fully re-entrant and also have file name and line for errors prior to the IOCCCMOCK ? Should I see about looking into this more? Last night, I must warn you, was truly awful, so I probably won't get much done today but that could of course change a bit later on. Not sure.

Purpose of this thread

The purpose of this thread, btw, is so we can keep track of things that still have to be done. This is different from the other thread in that that's just a task list of things that have to be done, whereas here we can discuss it and keep each other updated. Of course if you would like to do this with the other thread that's okay too but I decided to open this issue to keep the other one as clean and as short as possible.

So what do we actually have to do before a formal release? I can think of the following things. Please feel free to update and/or mark anything complete at any time. I am making it a task list but the idea is to actually make a new comment when each one is updated.

man pages / documentation

Specifically the man pages for jsemcgen.sh and jsemtblgen have to be written. I am in need of a better idea of what the JSON semantics tables are for in order to write these. I might look at the script invocations to see if I can get a better idea but it would be good if you could clarify in the other thread (man page one).

Once chkentry is finished we need to update the man page for that.

Finish chkentry

You said you're not clear yet on how it needs to work exactly so I've not worked on it since then. But once this is done I believe there are no other tools that have to be written so we can go checking for consistencies and typos and finishing man pages. I say 'we' but of course 'we' refers to me. :-)

JSON re-entrancy and/or better error reporting ?

I'm not sure if this needs to be done or not before IOCCCMOCK can run. Please advise.

Any server side stuff that I am unaware of that I cannot help with ? (I'm not sure if this will change the time when this repo can be formally released to the wider public)

NOTE: The above check item was answered elsewhere and is out of scope of this issue.

Anything else unrelated to the server ?

NOTE: The above check item was answered elsewhere and is out of scope of this issue.

BTW: We can certainly open new issues or use other issues already open for any item. This is just a simple tracker with the possibility of being able to discuss it and keep each other updated (assuming of course we both remember to do so! :-) )

Modify mkiocccentry(1) to indicate that the submission server is not ready (even for IOCCCMOCK).

During the phase where the commenting and testing mkiocccentry(1) (and the tools it calls), we need to modify mkiocccentry(1) to NOT refer to any submission server (in the pre-IOCCCMOCK pubic review phase). We can indicate that there is NOT YET an IOCCC submission server, that the IOCCC is NOT open for submissions, and to PLEASE DO NOT SEND the IOCCC judges any IOCCC entries at this time.

We probably want to have some sort of:

#undef IOCCC_SUBMIT_SERVER_READY
#if defined(IOCCC_SUBMIT_SERVER_READY)
... stuff about, after registering for the IOCCC, uploading the compressed tarball to the submit server ...
#else /* IOCCC_SUBMIT_SERVER_READY */
... the the submit server is NOT ready, the IOCCC is NOT open, please do NOT send the IOCCC your entry ...
#endif /* IOCCC_SUBMIT_SERVER_READY */

Enhancement: Replace DEBUG_LINT with use of attribute((format(printf ...)))

The need for a DEBUG_LINT macro set is a holdover from back in the late 1980's.

We can replace the need for DEBUG_LINT by using __attribute__((format(printf ...))) instead. For example:

extern void msg(const char *fmt, ...) \
        __attribute__((format(printf, 1, 2)));          /* 1=format 2=params */
extern void dbg(int level, const char *fmt, ...) \
        __attribute__((format(printf, 2, 3)));          /* 2=format 3=params */
extern void warn(const char *name, const char *fmt, ...) \
        __attribute__((format(printf, 2, 3)));          /* 2=format 3=params */
extern void warnp(const char *name, const char *fmt, ...) \
        __attribute__((format(printf, 2, 3)));          /* 2=format 3=params */
extern void err(int exitcode, const char *name, const char *fmt, ...) \
        __attribute__((noreturn)) __attribute__((format(printf, 3, 4)));        /* 3=format 4=params */
extern void errp(int exitcode, const char *name, const char *fmt, ...) \
        __attribute__((noreturn)) __attribute__((format(printf, 3, 4)));        /* 3=format 4=params */
extern void vfprintf_usage(int exitcode, FILE *stream, const char *fmt, ...) \
         __attribute__((format(printf, 3, 4)));                                 /* 3=format 4=params */

We are aware of the formatted error messages err() and friends. That question about their use will appear in a later enhancement issue.

Comments and questions welcome.

Enhancement: Write a test suite for `txzchk`

I just wrote the following to help start this but I also have some additional thoughts to add that might be considered to make it easier to test the tool. It would require that I make some changes to the tool (and possibly fnamchk) but I'm okay with that (though maybe you would want to do the changes to fnamchk):

--

`test_txzchk`: text files for a test suite for `txzchk`

The subdirectories in this directory have text files for a test suite for the
txzchk tool.

The good/ subdirectory has files that txzchk MUST report as valid.

The bad/ subdirectory has files that txzchk MUST report as invalid.

The files MUST end with the extension .txt and the filename MUST follow the
rules of the fnamchk tool.

The typical format is that which would be generated from tar -tJvf but
importantly these files are NOT tarballs: they are text files that would
have the output of the tar command. This is to make it much easier to add test
cases and it also prevents having the need to have tarballs in the repo (as well
as constructing said tarballs with invalid input that might be submitted due to
abuse).

Here's an example good file:

drwxr-xr-x  0 501    20          0 Jun  3 05:50 test-0/
-rw-r--r--  0 501    20       1854 Jun  3 05:50 test-0/Makefile
-rw-r--r--  0 501    20          4 Jun  3 05:50 test-0/extra2
-rw-r--r--  0 501    20          0 Jun  3 05:50 test-0/prog.c
-rw-r--r--  0 501    20       2793 Jun  3 05:50 test-0/.author.json
-rw-r--r--  0 501    20       3715 Jun  3 05:50 test-0/remarks.md
-rw-r--r--  0 501    20       1429 Jun  3 05:50 test-0/.info.json
-rw-r--r--  0 501    20          4 Jun  3 05:50 test-0/extra1

Let's say the file is called entry.test-0.1654260612.txt. We can run the
following command to test txzchk on it:

$ ./txzchk -T -E txt entry.test-0.1654260612.txt

This will report on whether or not the tarball is valid. If you want to not see
any output at all and only check the exit status one can do:

$ ./txzchk -T -E txt entry.test-0.1654260612.txt 2>&1 >/dev/null || echo invalid

This will not show anything as the tarball is actually valid.

Enhancement: reduce the top level directory complexity for someone who just wants to submit to the IOCCC

Based in this excellent comment comment 1242818320 we offer the following suggestion:

The basic idea is to push stuff into sub-directories and leave only mandatory repo files and core of the tools that people entering the IOCCC might use. While this would not be done right way .. if something like this is implemented it could be done just prior to packaging for public input / IOCCCMOCK.

For example, leave only the top with the required repo files:

README.md
CODE_OF_CONDUCT.md
LICENSE

Plus this common repo file:

CHANGES.md

and these mkiocccentry related files:

Makefile
Makefile.example
mkiocccentry.c
mkiocccentry.h
mkiocccentry.1
iocccsize.c
iocccsize.h
iocccsize.1
chkentry.c
chkentry.h
chkentry.1

And these hidden files:

.gitignore
.editorconfig
.shellcheckrc
etc.

and perhaps these sub-directories:

dbg
dyn_array
json
man

This is just an idea. There are considerations such:

Should chkentry be at the top level .. perhaps?
Should ALL of the other man pages be pushed into that directory .. perhaps not?
Should there be a A_README_FIRST.txt file (or some such name) to give someone entering the IOCCC a quick start .. perhaps so
Should there be a FAQ.txt file .. perhaps so
Should some of the Makefile complexity could be moved into lower level Makefile(s) .. perhaps so?
etc.

In other words, push the repo complexity down one directory level AND make it easier for someone entering the IOCCC to QuickStart.

This is just a concept to be considered and discussed in this issue. It is possible that nothing will be done and this issue should be closed without any modifications. On the other hand there may be enough merit to make some changes to the structure of this repo. Such changes can be discussed in this issue.

If we did all of this, we might reduce the top level directory to someone that looks less intimidating .. here is quick mockup:

-r--r--r--  1 chongo  staff    7917 Sep 10 18:10 A_README_FIRST.txt
-r--r--r--  1 chongo  staff   10876 Sep 10 18:08 CHANGES.md
-r--r--r--  1 chongo  staff    5235 Sep 10 18:08 CODE_OF_CONDUCT.md
-r--r--r--  1 chongo  staff    8592 Sep 10 18:11 FAQ.txt
-r--r--r--  1 chongo  staff    2815 Sep 10 18:08 LICENSE
-r--r--r--  1 chongo  staff   40791 Sep 10 18:08 Makefile
-r--r--r--  1 chongo  staff    1856 Sep 10 18:08 Makefile.example
-r--r--r--  1 chongo  staff    3193 Sep 10 18:08 README.md
-r--r--r--  1 chongo  staff    2724 Sep 10 18:19 chkentry.1
-r--r--r--  1 chongo  staff   16487 Sep 10 18:19 chkentry.c
-r--r--r--  1 chongo  staff    3216 Sep 10 18:19 chkentry.h
drwxr-xr-x  2 chongo  staff      64 Sep 10 18:09 dbg
drwxr-xr-x  2 chongo  staff      64 Sep 10 18:09 dyn_array
-r--r--r--  1 chongo  staff    2387 Sep 10 18:08 iocccsize.1
-r--r--r--  1 chongo  staff    6338 Sep 10 18:08 iocccsize.c
-r--r--r--  1 chongo  staff    2124 Sep 10 18:09 iocccsize.h
drwxr-xr-x  2 chongo  staff      64 Sep 10 18:09 json
drwxr-xr-x  2 chongo  staff      64 Sep 10 18:09 man
-r--r--r--  1 chongo  staff    4222 Sep 10 18:09 mkiocccentry.1
-r--r--r--  1 chongo  staff  153126 Sep 10 18:09 mkiocccentry.c
-r--r--r--  1 chongo  staff    9459 Sep 10 18:09 mkiocccentry.h

So someone looking into the top level directory might only see:

A_README_FIRST.txt LICENSE            chkentry.1         dyn_array          json           mkiocccentry.h
CHANGES.md         Makefile           chkentry.c         iocccsize.1        man
CODE_OF_CONDUCT.md Makefile.example   chkentry.h         iocccsize.c        mkiocccentry.1
FAQ.txt            README.md          dbg                iocccsize.h        mkiocccentry.c

Such a directory layout might look a lot less intimidating.

Of course doing just some this would involve substantial changes to the Makefile, #include lines, etc.

Enhancement: replace /NOTREACHED/ with feature macros.

Comments of the form:

        /*NOTREACHED*/

linger from the old days of link and pre-gcc compilers such as K&R. Modern compilers do not honor those comments, as was seen in issue #19 .

For example code such as:

    if (ret < 0) {
        errp(2, __func__, "gettimeofday failed");
        /*NOTREACHED*/
    }

does not work.

Perhaps macro such as the following could be defined in dbg.h (because dbg.c and other C source have need of it):

#if __has_builtin(__builtin_unreachable)
#    define NOTREACHED __builtin_unreachable()
#else
#    define NOTREACHED abort()
#endif /* __has_builtin(__builtin_unreachable) */

So such code as above could be re-written as:

    if (ret < 0) {
        errp(2, __func__, "gettimeofday failed");
        NOTREACHED;
    }

There is also the possibility of using __attribute__((noreturn)) as in:

extern void err(int exitcode, const char *name, const char *fmt, ...) __attribute__((noreturn));
extern void errp(int exitcode, const char *name, const char *fmt, ...) __attribute__((noreturn));

That would address the majority of there cases where /*NOTREACHED*/ was being used.

Both clang and gcc seem to have __attribute__((noreturn)). For those compilers who do not have the noreturn attribute, the NOTREACHED macro would apply. And in the case where the compiler does not even have __builtin_unreachable, the call to abort would be available.

Comments? Suggestions?

Bug: jstrdecode objects to NUL bytes

Describe the bug
The jstrdecode tool exits non-zero when it fines a NUL byte.

To Reproduce
Steps to reproduce the behavior (adjust steps as needed):
0. echo -n '"NULXbyte"' | tr 'X' '\0' | hd

echo -n '"NULXbyte"' | tr 'X' '\0' | ./jstrencode
echo -n '"NULXbyte"' | tr 'X' '\0' | ./jstrencode | ./jstrdecode -n
echo -n '"NULXbyte"' | tr 'X' '\0' | ./jstrencode | ./jstrdecode -n 2>/dev/null | hd

Expected behavior
The jstrdecode should decode the JSON "NUL\u0000byte" string.

Step 2 should not exit non-zero.

Step 3 should output the same as step 0.

Output
$ echo -n '"NULXbyte"' | tr 'X' '\0' | ./jstrencode | ./jstrdecode
Warning: json_decode: found non--escaped char: 0x0a
Warning: jstrdecode_stdin: error while encoding stdin buffer

zsh: done echo -n '"NULXbyte"' | tr 'X' '\0' | ./jstrencode |
zsh: exit 1 ./jstrdecode

Environment (please complete the following information, or write n/a where not appliable):

OS: macOS
OS distribution: Apple
OS Version: 12.31.1
Compiler: n/a
Compiler version: n/a
Bison version: n/a
Flex verson: n/a
Shell: zsh
Airspeed Velocity of an Unladen Swallow: 42

Additional context
While jstrdecode might warn about NUL bytes, it should not exit non-zero when it finds one.

Bug: dyn_test segfaults under CentOS

I was going to see about fixing it but I decided (after a bit of time after waking up a bit more) that it is better that I work on the JSON parser (and discuss it as well) so I'm leaving this bug report instead.

Describe the bug

Segfaults.

To Reproduce

Steps to reproduce the behavior (adjust steps as needed):

Just run ./dyn_test (or run make test which does this).

Expected behavior

Should not segfault.

Output

With debug output:

debug[11]: dyn_array_append_value(array, buf, value): in-place: allocated: 1000448 elements of size: 8 in use: 999997
debug[11]: dyn_array_append_value(array, buf, value): in-place: allocated: 1000448 elements of size: 8 in use: 999998
debug[11]: dyn_array_append_value(array, buf, value): in-place: allocated: 1000448 elements of size: 8 in use: 999999
debug[11]: dyn_array_append_value(array, buf, value): in-place: allocated: 1000448 elements of size: 8 in use: 1000000
debug[11]: dyn_array_grow(array, 1000448): moved: allocated: 2000896 elements of size: 8 in use: 1000000
Segmentation fault (core dumped)

I ran it under the debugger both running the program as well as looking at the core. Here's a stacktrace:

(gdb) bt
#0  0x00007f4bc400b6a6 in ?? ()
#1  0x0000000000408c9b in dyn_array_append_set (array@entry=<optimized out>, array_to_add_p@entry=<optimized out>, 
    count_of_elements_to_add@entry=<optimized out>) at dyn_array.c:422
#2  0x0000000000408f1c in dyn_array_concat_array (array@entry=<optimized out>, other@entry=<optimized out>) at dyn_array.c:526
#3  0x000000000040166f in main (argc@entry=<optimized out>, argv@entry=<optimized out>) at dyn_test.c:138
(gdb) up
#1  0x0000000000408c9b in dyn_array_append_set (array@entry=<optimized out>, array_to_add_p@entry=<optimized out>, 
    count_of_elements_to_add@entry=<optimized out>) at dyn_array.c:422
422	    memcpy(p, array_to_add_p, array->elm_size * count_of_elements_to_add);
(gdb) up
#2  0x0000000000408f1c in dyn_array_concat_array (array@entry=<optimized out>, other@entry=<optimized out>) at dyn_array.c:526
526	    moved = dyn_array_append_set(array, other->data, other->count);

Now as for the pointers:

(gdb) p array
$1 = (struct dyn_array *) 0xa3d010
(gdb) p *array
$2 = {
  elm_size = 8, 
  zeroize = true, 
  count = 1000000, 
  allocated = 2000896, 
  chunk = 1024, 
  data = 0x7f4bc27a5010
}
(gdb) p other 
$3 = (struct dyn_array *) 0xa3d010
(gdb) p *other
$4 = {
  elm_size = 8, 
  zeroize = true, 
  count = 1000000, 
  allocated = 2000896, 
  chunk = 1024, 
  data = 0x7f4bc27a5010
}

I'm happy to gather other information. I do have a fair bit of experience with gdb though it's been a while since I've used it a great deal.

Environment (please complete the following information, or write n/a where not appliable):

For system information that should be relevant:

OS:

$ cat /etc/centos-release
CentOS Linux release 7.9.2009 (Core)

As for version of compiler:

$ cc --version|head -n1
cc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-44)

And now I hope to look at the json thread! We shall see how far I get though.

Enhancement: Discuss man pages (+ other documentation) that have yet to be added (and those already added)

I wasn't sure if this should be in the 'issues that aren't really issues' or not but I decided to make it a new issue for now as maybe there's enough to discuss that it warrants a new issue.

I think I know the answer but does every script need a man page? Every tool should have one but what about every script? Do we need any other man pages that aren't section 1?

Currently (as of 7 July 2022) running

$ for i in $(ls -1 *.{1,3}); do echo "- ${i}";done

(Maybe there's a more efficient way to do this but this is what came to my mind at 0223 ... but suggestions welcome please!)

Will give you:

chkentry.1
dbg.3
fnamchk.1
iocccsize.1
jparse.1
jstrdecode.1
jstrencode.1
limit_ioccc.1
mkiocccentry.1
old.chkauth.1
old.chkinfo.1
txzchk.1
utf8_test.1
verge.1

Now chkentry.1 is not complete because the tool is not yet written. The old.chk* will only be used in some concepts: the tools are no longer needed as chkentry will take their place.

As I said at the time I wasn't sure if limit_ioccc.sh's man page should be section 1 but I decided to make it that way because it can be used (and is used) in shell scripts via source (or .).

But there are many man pages that are missing too. What should be added? Should anything be changed in the man pages that exist?

I am very well aware of course that even if new discussion brings changes to any of these that prior to the repo being tested in the general public there might be some changes that are needed. Perhaps this issue should be left open until that is done? Certainly until all tools have man pages though. As new tools are added we can add more to discuss.

Maybe there's an easy way to come up with a list of missing man pages - I can possibly think of one - but I am trying to make this quick and discussing man pages that exist is also good.

BTW: I know I asked before and the answer then was 'no' but should there be html versions of the man pages at any point? It might be easy to convert to html via not using the -P option and redirecting to markdown files ... though of course maybe we'd have to make some manual edits. Or perhaps there's even a man2html ? I seem to recall there might be. Just a thought that might be worth thinking about on the website: or perhaps not. I leave that up to you but I'd be happy to help with it if you decide this is a good idea.

Going to try resting now .. hope you're having a nice sleep my good friend! I do have to be out later on today - post surgery doctor visit - but at that point I would be done here for the day anyway most likely. Did a small pull request. Not sure if I'll do more later on. We shall see. BCNU! :)

typos in cp/ls check error messages

"    mkiocccentry -t cp ...",
"    mkiocccentry -t cp ...",
"    mkiocccentry -c ls ...",
"    mkiocccentry -t ls ...",
"    mkiocccentry -t ls ...",

Should be:

"    mkiocccentry -c cp ...",
"    mkiocccentry -c cp ...",
"    mkiocccentry -l ls ...",
"    mkiocccentry -l ls ...",
"    mkiocccentry -l ls ...",

Bug: Allow NUL bytes in JSON

Describe the bug
In some cases, NUL bytes are disallowed in the JSON parser.

To Reproduce
Steps to reproduce the behavior (adjust steps as needed):

echo -n '"NUL byte"' | ./jparse -J1 -
echo -n '"NULXbyte"' | tr 'X' '\0' | hd
echo -n '"NULXbyte"' | tr 'X' '\0' | ./jparse -J1 -

Expected behavior
While command 1 works, and command 2 shows the NUL byte is being fed to stdin of jparse,
command 3 results in an error and parse exits non-zer0.

Screenshots
$ echo -n '"NULXbyte"' | tr 'X' '\0' | ./jparse -J1 -
JSON DEBUG[1]: Calling parse_json_file("-"):
Warning: parse_json_file: found embedded NUL byte in stdin
zsh: done echo -n '"NULXbyte"' | tr 'X' '\0' |
zsh: exit 1 ./jparse -J1 -

Environment (please complete the following information, or write n/a where not appliable):

OS: macOS
OS distribution: Apple
OS Version: 12.31.1
Compiler: n/a
Compiler version: n/a
Bison version: 3.8.2
Flex verson: 2.6.4
Shell: zsh,
Airspeed Velocity of an Unladen Swallow: African or European?

Additional context
It seems to be OK to warn about NUL bytes, but a struct reading of ECMA-404 suggests that there is no prohibition against having a NUL byte in JSON.

NOTE: When we say NUL byte, we do not refer to the two byte sequence ASCII backslash followed by ASCII zero (i.e., not \0), we refer to a single byte with a numeric value of 0.

BTW: The only place were a NUL byte appears to her allowed is inside a JSON string. A NUL byte outside of a JSON string will be a syntax error. Nevertheless a check should be made that a NUL byte outside of a JSON string will be detected and marked as a syntax error.

Enhancement: Add a wrapper around mkiocccentry calls to system() and popen() to handle a wider set of args

Add a function that convert a shell string used by mkiocccentry calls to system() and popen() to handle arguments, such as filenames, that have characters such as spaces, *'s and/or have leading -'s.

Bug: CHANGES.md is out of date

Describe the bug
The CHANGES.md needs to be updated to describe some of the more recent changes

To Reproduce
Steps to reproduce the behavior:

cat CHANGES.md

Expected behavior
A few of the more recent major changes should be listed.

Screenshots
n/a

Desktop (please complete the following information):

OS: macOS
OS distribution: Apple
OS Version: 12.3.1
Compiler: n/a

Additional context
Perhaps it is time to call the current state "Release 0.2"?

Please put the NEW release notes ABOVE the previous release notes.

Question: Issues that aren’t really major issues but are still issues

This is for relevant to what is being worked on (like the JSON parser issue #156) but also some OT things. The purpose of this is to hopefully make it a little bit easier to find more relevant information in the other issues as already they’re very long and there are some comments that possibly could have been here instead.

There might be a better way to go about this. If so I am open to that but I think this might be a good idea. Of course we have to make use of it and discern the time to use it. Perhaps more than one of these could be made for the different tools: don’t know.

An example where it might be useful: sometimes I put an update on the day in the other issue but that doesn’t have to be there. Lately I have tried to do this on pull requests but there are still times where I don’t.

Hopefully this helps make the other issues (currently only #156) more clear and concise.

Enhancement: create the chkentry tool

We can discuss the details here. If you've not responded to the question on . in the other thread before I get a chance to (too tired right now) I will copy and paste my reply here to discuss it here. Otherwise I'll move my reply to your new reply here.

Please add the tags you want and also assign it to me. You can then close the other issues and refer to this one here as you suggested.

I don't know when I can start working on this but the first part will be writing main() and a sanity check function. This most likely will not happen today I'm afraid. Not even sure if I'll get anything done today here. I'm hoping I can get some rest soon but last time I tried earlier this morning I couldn't.

Later on I will wake up more but whether I do more here is yet to be seen. Anyway we can now discuss the new tool here which should also help remove some clutter elsewhere (or prevent further clutter I guess).

Use tar options to disable including of the username/group in the archive.

Check the binary content of these archives:

$ tar -cf test.tar README.md
$ hexdump -C test.tar

$ tar --numeric-owner --owner=0 --group=0 -cf test.tar README.md
$ hexdump -C test.tar

# aliases: --old-archive, --portability, --format=v7
$ tar --format=v7 -cf test.tar README.md 
$ hexdump -C test.tar

flex/bison scripts not regenerating files properly

At first I didn't think this was an issue but I think there's a problem with the scripts for flex and bison. Look at this:

$ git status
On branch updates
Your branch is up to date with 'origin/updates'.

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
	modified:   jparse.l
	modified:   jparse.y
	modified:   sorry.tm.ca.h

This was after running make parser and note that the C files have not been updated and neither have the header files. I'm not sure if this was introduced when I modified the Makefile to not use -D or if it was introduced earlier but it's an obvious problem and it's also breaking a thought I had (which I was going to put into the issues that aren't really issues thread). Any idea what might be causing this? The output of make parser btw:

$ make parser 
rm -f jparse.tab.c jparse.tab.h
make jparse.tab.c jparse.tab.h
rm -f limit_ioccc.sh
grep -E '^#define (RULE_|MAX_|UUID_|MIN_|IOCCC_)' limit_ioccc.h | \
	    awk '{print $2 "=\"" $3 "\"" ;}' | tr -d '[a-z]()' | \
	    sed -e 's/"_/"/' -e 's/""/"/g' -e 's/^/export /' >> limit_ioccc.sh
grep -hE '^#define (.*_VERSION|TIMESTAMP_EPOCH)' version.h limit_ioccc.h | \
	    grep -v 'UUID_VERSION' | \
	    sed -e 's/^#define/export/' -e 's/ "/="/' -e 's/"[	 ].*$/"/' >> limit_ioccc.sh
if grep -q '^#define DIGRAPHS' limit_ioccc.h; then \
	    echo "export DIGRAPHS='yes'"; \
	else \
	    echo "export DIGRAPHS="; \
	fi >> limit_ioccc.sh
if grep -q '^#define TRIGRAPHS' limit_ioccc.h; then \
	    echo "export TRIGRAPHS='yes'"; \
	else \
	    echo "export TRIGRAPHS="; \
	fi >> limit_ioccc.sh
cc -D_DEFAULT_SOURCE -D_ISOC99_SOURCE -D_POSIX_C_SOURCE=200809L -D_XOPEN_SOURCE=600 -std=gnu11 -O0 -g -pedantic -Wall -Wextra -Werror -Wno-unused-command-line-argument -Wno-poison-system-directories -Wno-unreachable-code-break -Wno-padded -lm dyn_array.c -c
cc -D_DEFAULT_SOURCE -D_ISOC99_SOURCE -D_POSIX_C_SOURCE=200809L -D_XOPEN_SOURCE=600 -std=gnu11 -O0 -g -pedantic -Wall -Wextra -Werror -Wno-unused-command-line-argument -Wno-poison-system-directories -Wno-unreachable-code-break -Wno-padded -lm verge.c dbg.o util.o dyn_array.o -o verge
./run_bison.sh -B /opt/homebrew/opt/bison/bin -B /opt/homebrew/bin -B /opt/local/bin -B /usr/local/opt -B /usr/local/bin -B . -p jparse -v 1 -- -d
./run_bison.sh: debug[1]: args to pass to bison before jparse.y: -d
./run_bison.sh: debug[3]: checking for bison as: bison
./run_bison.sh: debug[1]: found a good bison: /opt/local/bin/bison
/opt/local/bin/bison -d jparse.y
# prepending comment and line number reset to jparse.tab.c
cat sorry.tm.ca.h > /var/folders/6n/nb4yym2d1r50sxlz7s0nt3dr0000gn/T/jparse.tab.c.XXXXXXXX.m3PqLqyB
cat jparse.tab.c >> /var/folders/6n/nb4yym2d1r50sxlz7s0nt3dr0000gn/T/jparse.tab.c.XXXXXXXX.m3PqLqyB
mv -v -f /var/folders/6n/nb4yym2d1r50sxlz7s0nt3dr0000gn/T/jparse.tab.c.XXXXXXXX.m3PqLqyB jparse.tab.c
/var/folders/6n/nb4yym2d1r50sxlz7s0nt3dr0000gn/T/jparse.tab.c.XXXXXXXX.m3PqLqyB -> jparse.tab.c
# completed update of jparse.tab.c
# prepending comment and line number reset to jparse.tab.h
cat sorry.tm.ca.h > /var/folders/6n/nb4yym2d1r50sxlz7s0nt3dr0000gn/T/jparse.tab.h.XXXXXXXX.QNJvzzF4
cat jparse.tab.h >> /var/folders/6n/nb4yym2d1r50sxlz7s0nt3dr0000gn/T/jparse.tab.h.XXXXXXXX.QNJvzzF4
mv -v -f /var/folders/6n/nb4yym2d1r50sxlz7s0nt3dr0000gn/T/jparse.tab.h.XXXXXXXX.QNJvzzF4 jparse.tab.h
/var/folders/6n/nb4yym2d1r50sxlz7s0nt3dr0000gn/T/jparse.tab.h.XXXXXXXX.QNJvzzF4 -> jparse.tab.h
# completed update of jparse.tab.h
./run_bison.sh: debug[1]: -rw-------  1 cody  staff  69558 Apr 25 05:37 jparse.tab.c
./run_bison.sh: debug[1]: -rw-------  1 cody  staff  5446 Apr 25 05:37 jparse.tab.h
./run_bison.sh: debug[1]: bison run OK: formed jparse.tab.c and jparse.tab.h
make[1]: `jparse.tab.h' is up to date.
rm -f jparse.c
make jparse.c
./run_flex.sh -F /opt/homebrew/opt/flex/bin -F /opt/homebrew/bin -F /opt/local/bin -F /usr/local/opt -F /usr/local/bin -F . -p jparse -v 1 -- -d -8 -o jparse.c
./run_flex.sh: debug[1]: args to pass to flex before jparse.y: -d -8 -o jparse.c
./run_flex.sh: debug[3]: checking for flex as: flex
./run_flex.sh: debug[1]: found a good flex: /opt/local/bin/flex
/opt/local/bin/flex -d -8 -o jparse.c jparse.l
# prepending comment and line number reset to jparse.c
cat sorry.tm.ca.h > /var/folders/6n/nb4yym2d1r50sxlz7s0nt3dr0000gn/T/jparse.c.XXXXXXXX.t1CpxXrK
cat jparse.c >> /var/folders/6n/nb4yym2d1r50sxlz7s0nt3dr0000gn/T/jparse.c.XXXXXXXX.t1CpxXrK
mv -v -f /var/folders/6n/nb4yym2d1r50sxlz7s0nt3dr0000gn/T/jparse.c.XXXXXXXX.t1CpxXrK jparse.c
/var/folders/6n/nb4yym2d1r50sxlz7s0nt3dr0000gn/T/jparse.c.XXXXXXXX.t1CpxXrK -> jparse.c
# completed update of jparse.c
./run_flex.sh: debug[1]: -rw-------  1 cody  staff  65491 Apr 25 05:37 jparse.c
./run_flex.sh: debug[1]: flex run OK: formed jparse.c

so it appears that it's being formed but it's not being updated.

Enhancement: create the jauthchk tool - check on the contents of an .author.json file

We need to create the jauthchk tool in order to help verify that contents of an .author.json file found within an entry directory.

This tool will primarily be used by other tools (not humans). As such it should behave like fnamchk in that if all is well, it should not print anything and simply exit 0. If there are problems found with the .author.json file, then warning messages should be printed to stderr AND the jauthchk tool should exit with a non-zero status. The use of a -v level may be use to assist in debugging.

The jauthchk tool is primarily a stand alone tool. As a sanity check, the mkiocccentry program should execute the jauthchk code AFTER .author.json file has been created and before the compressed tarball is formed. If mkiocccentry program sees a 0 exit status, then all is well. For a non-zero exit code, the tool probably should abort because any problems detected by jauthchk based on what mkiocccentry wrote into .author.json indicates there is a serious mismatch between what mkiocccentry is doing and what jauthchk expects.

The following might be how mkiocccentry output is changed with the use of this tool (and the other tool):

Is the above list a correct list of files in your entry? [yn]: y

Checking the format of .info.json ...
... all appears well with the .info.json file.

Checking the format of .author.json ...
... all appears well with the .author.json file.

About to run the tar command to form the compressed tarball ...

As a stand alone tool, the jauthchk tool will be invoked by other tools as part of the IOCCC submission process. That process is beyond the scope of this repo. Suffice it to sat the the IOCCC judges will use this tool is part of their submission workflow.

Here is a possible command line usage message:

jauthchk [-h] [-v level] [-V] file

	-h			print help message and exit 0
	-v level		set verbosity level: (def level: 0)
	-V			print version string and exit

        file                    path to a .author.json file

exit codes:

        0                       no errors or warnings detected
        >0                      some error(s) and/or warning(s) were detected

NOTE: We mention file above even though the canonical filename will be .author.json. The tool should NOT check, nor object to using a different filename.

The mkiocccentry tool will need to invoke this tool. As such a similar method used to find and specify the location of txzchk should be used. As this tool is one of 2 tools being considered, we recommend the following of added to the mkiocccentry command line:

	-j /path/to/jinfochk	path to jinfochk executable used by txzchk (def: ./jinfochk)
	-J /path/to/jauthchk	path to jauthchk executable used by txzchk (def: ./jauthchk)

IMPORTANT: While it might be tempting to consider depending on some general JSON checker, we do NOT need nor want that. It is important that the mkiocccentry GitHub repo remain stand alone. I.e., all the code needed by someone wishing to enter the IOCCC (beside a C compiler, make, tar, cp, ls) should found in this GitHub repo alone. As there is NO standard JSON tool in widespread distribution the all of the code for this tool needs to reside in this repo only.

IMPORTANT: We do not need a general JSON format checker. We only need to verify that the file contains the JSON needed and only the JSON needed for the judges to process IOCCC entries.

While is it NOT recommended, if someone wishes to edit their .author.json and re-create the compressed tarball we cannot stop them. As such mkiocccentry should be STRICT on what is writes into .author.json AND jauthchk should be permissive (but not to a fault) in what is considers as OK.

This tool should neither generate an error, nor warn if someone were to reformat the JSON. And as JSON is not order dependent, of someone wishes to reorder the JSON elements, that is fine. As long as all the requirement JSON elements are present, and no new JSON elements are found, and the version string matches, all is OK.

Should something go wrong and a change to the JSON is required during an open IOCCC, the judges will preserve the older JSON check tools and use those against older JSON formats. This there is no need for a >= version check: a version string match seems good enough.

See the a followup comment for details on the checks needed against an .author.json file.

Enhancement: create the jinfochk tool - check on the contents of an .info.json file

We need to create the jinfochk tool in order to help verify that contents of an .info.json file found within an entry directory.

This tool will primarily be used by other tools (not humans). As such it should behave like fnamchk in that if all is well, it should not print anything and simply exit 0. If there are problems found with the .info.json file, then warning messages should be printed to stderr AND the jinfochk tool should exit with a non-zero status. The use of a -v level may be use to assist in debugging.

The jinfochk tool is primarily a stand alone tool. As a sanity check, the mkiocccentry program should execute the jinfochk code AFTER .info.json file has been created and before the compressed tarball is formed. If mkiocccentry program sees a 0 exit status, then all is well. For a non-zero exit code, the tool probably should abort because any problems detected by jinfochk based on what mkiocccentry wrote into .info.json indicates there is a serious mismatch between what mkiocccentry is doing and what jinfochk expects.

The following might be how mkiocccentry output is changed with the use of this tool (and the other tool):

Is the above list a correct list of files in your entry? [yn]: y

Checking the format of .info.json ...
... all appears well with the .info.json file.

Checking the format of .author.json ...
... all appears well with the .author.json file.

About to run the tar command to form the compressed tarball ...

As a stand alone tool, the jinfochk tool will be invoked by other tools as part of the IOCCC submission process. That process is beyond the scope of this repo. Suffice it to sat the the IOCCC judges will use this tool is part of their submission workflow.

Here is a possible command line usage message:

jinfochk [-h] [-v level] [-V] file

	-h			print help message and exit 0
	-v level		set verbosity level: (def level: 0)
	-V			print version string and exit

        file                    path to a .info.json file

exit codes:

        0                       no errors or warnings detected
        >0                      some error(s) and/or warning(s) were detected

NOTE: We mention file above even though the canonical filename will be .info.json. The tool should NOT check, nor object to using a different filename.

The mkiocccentry tool will need to invoke this tool. As such a similar method used to find and specify the location of txzchk should be used. As this tool is one of 2 tools being considered, we recommend the following of added to the mkiocccentry command line:

	-j /path/to/jinfochk	path to jinfochk executable used by txzchk (def: ./jinfochk)
	-J /path/to/jauthchk	path to jauthchk executable used by txzchk (def: ./jauthchk)

IMPORTANT: While it might be tempting to consider depending on some general JSON checker, we do NOT need nor want that. It is important that the mkiocccentry GitHub repo remain stand alone. I.e., all the code needed by someone wishing to enter the IOCCC (beside a C compiler, make, tar, cp, ls) should found in this GitHub repo alone. As there is NO standard JSON tool in widespread distribution the all of the code for this tool needs to reside in this repo only.

IMPORTANT: We do not need a general JSON format checker. We only need to verify that the file contains the JSON needed and only the JSON needed for the judges to process IOCCC entries.

While is it NOT recommended, if someone wishes to edit their .info.json and re-create the compressed tarball we cannot stop them. As such mkiocccentry should be STRICT on what is writes into .info.json AND jinfochk should be permissive (but not to a fault) in what is considers as OK.

This tool should neither generate an error, nor warn if someone were to reformat the JSON. And as JSON is not order dependent, of someone wishes to reorder the JSON elements, that is fine. As long as all the requirement JSON elements are present, and no new JSON elements are found, and the version string matches, all is OK.

Should something go wrong and a change to the JSON is required during an open IOCCC, the judges will preserve the older JSON check tools and use those against older JSON formats. This there is no need for a >= version check: a version string match seems good enough.

See the a followup comment for details on the checks needed against an .info.json file.

Define exit codes for the tools

Based on the point you made from the recent commit maybe it's a good idea to discuss exit codes for all the tools. Obviously non-zero is one or more issues but what should they be?

I have some ideas but right now I'm too tired to really put them to words.

Maybe opening an issue a discussion can be had about it. After that the tools can be updated as well as the man pages.

I'll add my thoughts later on - I think it won't be until tomorrow.

Dead code and enabling -Wextra

I propose to enable "-Wextra" in the Makefile.

Among others, this will reveal several places of dead code that can be safely removed, such as (produced by GCC 8.3.0):

mkiocccentry.c:5774:14: warning: comparison of unsigned expression < 0 is always false [-Wtype-limits]
      if (len < 0) {
              ^

Moreover, this will hint to unused parameters, such as (produced by GCC 8.3.0), see #6:

mkiocccentry.c:5071:57: warning: unused parameter ‘entry_num’ [-Wunused-parameter]
 get_author_info(struct info *infop, char *ioccc_id, int entry_num, struct author **author_set_p)
                                                     ~~~~^~~~~~~~~

Warning: ‘strncat’ specified bound depends on the length of the source argument

The following warning is produced when compiled on GCC 8.3.0:

$ make
cc -O3 -g3 --pedantic -Wall mkiocccentry.c -o mkiocccentry
mkiocccentry.c: In function ‘write_info’:
mkiocccentry.c:6584:12: warning: ‘strncat’ specified bound depends on the length of the source argument [-Wstringop-overflow=]
     (void) strncat(infop->gmtime, p, strlen(p)-1); /* -1 to remove trailing newline */
            ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
mkiocccentry.c:6584:38: note: length computed here
     (void) strncat(infop->gmtime, p, strlen(p)-1); /* -1 to remove trailing newline */
                                      ^~~~~~~~~

Bug report: make depend (soup/fmt_depend.sh) not consistent with output across different systems

Describe the bug

As I noted in the other issue the problem is that under some systems the output still differs from others (like macOS). Also at issue is that the use of tr is not so portable (the unescaped backslash at the end that is .. see below).

To Reproduce

Pull the changes (or git clone)
Run make depend
Try it on different systems e.g. fedora and macOS: you'll notice different formatting. See attached files in follow up comments.

Expected behaviour

It should be the same output as that's the purpose of the script.

Relevant Output

See the git diff under fedora for the format changes. Note that I changed the upper case 'I' to lower case in the sed invocation in the make depend rule because it doesn't exist under fedora sed. However I'm not including that part of the diff as it's already going to be a long report.

diff --git i/Makefile w/Makefile
index d72db9e..2a9a3e5 100644
--- i/Makefile
+++ w/Makefile
@@ -1101,11 +1101,12 @@ depend: all soup/fmt_depend.sh
 ###############
 # make depend #
 ###############
 
 ### DO NOT CHANGE MANUALLY BEYOND THIS LINE
-utf8_posix_map.o: utf8_posix_map.c utf8_posix_map.h util.h dyn_array.h dbg.h limit_ioccc.h version.h
+utf8_posix_map.o: utf8_posix_map.c utf8_posix_map.h util.h dyn_array.h dbg.h limit_ioccc.h \
+	version.h
 foo.o: foo.c foo.h dbg.h oebxergfB.h
 jparse.o: jparse.c jparse.h dbg.h util.h dyn_array.h json_parse.h json_util.h jparse.tab.h
 jparse.tab.o: jparse.tab.c jparse.h dbg.h util.h dyn_array.h json_parse.h json_util.h jparse.tab.h
 dbg.o: dbg.c dbg.h
 util.o: util.c dbg.h util.h dyn_array.h limit_ioccc.h version.h
@@ -1129,28 +1130,31 @@ sanity.o: sanity.c sanity.h util.h dyn_array.h dbg.h location.h utf8_posix_map.h
 	version.h
 utf8_test.o: test/utf8_test.c test/../utf8_posix_map.h test/../util.h test/../dyn_array.h \
 	test/../dbg.h test/../limit_ioccc.h test/../version.h
 verge.o: verge.c verge.h dbg.h util.h dyn_array.h limit_ioccc.h version.h
 dyn_array.o: dyn_array.c dyn_array.h dbg.h
-dyn_test.o: test/dyn_test.c test/dyn_test.h test/../util.h test/../dyn_array.h test/../dbg.h
+dyn_test.o: test/dyn_test.c test/dyn_test.h test/../util.h test/../dyn_array.h test/../dbg.h \
+	test/../dbg.h test/../dyn_array.h
 dbg_test.o: dbg_test.c dbg.h
 jnum_chk.o: test/jnum_chk.c test/jnum_chk.h test/../dbg.h test/../util.h test/../dyn_array.h \
-	test/../json_parse.h test/../json_util.h test/../version.h
+	test/../dbg.h test/../json_parse.h test/../util.h test/../json_util.h test/../json_parse.h \
+	test/../version.h
 jnum_gen.o: jnum_gen.c jnum_gen.h dbg.h util.h dyn_array.h json_parse.h json_util.h limit_ioccc.h \
 	version.h
 jnum_test.o: test/jnum_test.c test/../json_parse.h test/../util.h test/../dyn_array.h test/../dbg.h
 json_util.o: json_util.c dbg.h json_parse.h util.h dyn_array.h json_util.h
 jparse_main.o: jparse_main.c jparse_main.h dbg.h util.h dyn_array.h jparse.h json_parse.h \
 	json_util.h jparse.tab.h
-entry_util.o: entry_util.c dbg.h util.h dyn_array.h entry_util.h version.h json_parse.h json_util.h \
-	json_sem.h limit_ioccc.h location.h
-jsemtblgen.o: jsemtblgen.c jsemtblgen.h dbg.h util.h dyn_array.h json_util.h json_parse.h jparse.h \
-	jparse.tab.h json_sem.h iocccsize.h
+entry_util.o: entry_util.c dbg.h util.h dyn_array.h entry_util.h version.h json_parse.h \
+	json_util.h json_sem.h limit_ioccc.h location.h
+jsemtblgen.o: jsemtblgen.c jsemtblgen.h dbg.h util.h dyn_array.h json_util.h json_parse.h \
+	jparse.h jparse.tab.h json_sem.h iocccsize.h
 chk_sem_auth.o: soup/chk_sem_auth.c soup/chk_sem_auth.h soup/../json_sem.h soup/../util.h \
 	soup/../dyn_array.h soup/../dbg.h soup/../json_parse.h soup/../json_util.h
 chk_sem_info.o: soup/chk_sem_info.c soup/chk_sem_info.h soup/../json_sem.h soup/../util.h \
 	soup/../dyn_array.h soup/../dbg.h soup/../json_parse.h soup/../json_util.h
 chk_validate.o: soup/chk_validate.c soup/chk_validate.h soup/../entry_util.h soup/../version.h \
 	soup/../json_parse.h soup/../util.h soup/../dyn_array.h soup/../dbg.h soup/../json_util.h \
-	soup/../json_sem.h soup/../limit_ioccc.h soup/../location.h soup/chk_sem_auth.h soup/chk_sem_info.h
+	soup/../json_sem.h soup/../limit_ioccc.h soup/../location.h soup/../json_util.h soup/../json_sem.h \
+	soup/chk_sem_auth.h soup/chk_sem_info.h
 json_sem.o: json_sem.c dbg.h json_sem.h util.h dyn_array.h json_parse.h json_util.h
 have_timegm.o: have_timegm.c

Running make depend under macOS does not have this problem. Also of note is that under fedora you'll see:

tr: warning: an unescaped backslash at end of string is not portable

On that note see the following URL with information on the changes. Seems POSIX is not entirely clear (according to the comment in the patch): https://lists.gnu.org/archive/html/bug-coreutils/2007-07/msg00082.html. Changing the \\ to be \\\\ seems to work though the same problem that I am reporting still occurs.

bug report attachment

bug-report.20221106.005949.txt

Please provide the following information:

Did you attach the above requested bug report file? [yes, no & why not]

Of course! How else could I shameless (or is that shamefully ? :-) ) promote my own script ?! :-)

What is the airspeed Velocity of an Unladen Swallow? [n/a, :-), X m/s, unknown, etc.]

If you don't know a number the answer must be 42! :-)

Additional context

Just the comments I posted in the other thread about hostchk.sh. I might normally paste the links here but I think this is enough as I think I covered everything. Can make a new comment if necessary.

Bug: Zero length JSON documents should not allowed

Describe the bug
The JSON parser appears to allow for an empty JSON document

To Reproduce
Steps to reproduce the behavior:

./jparse -J1 -s ""

Expected behavior
The parser declares the JSON document to be invalid

Screenshots

./jparse -J1 -s ""
JSON DEBUG[1]: Calling parse_json_block(""):
Warning: parse_json_block: passed empty string

Desktop (please complete the following information):

OS: macOS
Browser: safari
Version: 12.3.1

Additional context
While the parser generates a warning, it should declare the document to be in error.

Bug report: even some modern systems need feature test macro(s) for some functions

Describe the bug

As I noted in comment #396 (comment) even modern systems require a feature test macro. This is really unfortunate because in the Makefile we have:

# While this repo does not officially support pre-c11 systems that have
# problems with the timegm() function not being declared in <time.h>.
#
# If your pre-c11 system fails to compile this code, we apologize and
# request that you compile this repo on a more up to date system such as
# a system that fully support c11 or later.
#
# XXX - ####################################################################### - XXX #
# XXX - In 2024 we will D_LEGACY and stop trying to support such legacy systems - XXX #
# XXX - ####################################################################### - XXX #
#
D_LEGACY+= -D_DEFAULT_SOURCE -D_ISOC99_SOURCE -D_POSIX_C_SOURCE=200809L -D_XOPEN_SOURCE=600 -DTIMEGM_PROBLEM

Fedora 36 is not pre-c11 and definitely not legacy. It does support strptime(3) but the synopsis is:

SYNOPSIS
       #define _XOPEN_SOURCE       /* See feature_test_macros(7) */
       #include <time.h>

       char *strptime(const char *restrict s, const char *restrict format,
                      struct tm *restrict tm);

It gets more curious but I'll defer that for further context.

To Reproduce

Steps to reproduce the behaviour (adjust steps as needed):

Try running the ./have_timegm.sh script and you'll see that it reports it having a problem.

Expected behaviour

Under a modern system it should not actually be reporting there is a problem. It's doing so because of the feature test macro. But see the relevant output for more details on this as there is actually a way to make it work (not sure on all systems though)!

Relevant Output

So here's a couple contradictory outputs. First the script itself:

$ ./have_timegm.sh 
-DTIMEGM_PROBLEM
$ echo $?
1

Now let's try the compilation via how it's done with the script:

$ cc -std=gnu11 -Wall -Wextra -Werror -pedantic -o have_timegm have_timegm.c
have_timegm.c: In function 'main':
have_timegm.c:70:11: error: implicit declaration of function 'strptime'; did you mean 'strftime'? [-Werror=implicit-function-declaration]
   70 |     ptr = strptime(time_string, FMT, &timeptr);
      |           ^~~~~~~~
      |           strftime
have_timegm.c:70:9: error: assignment to 'char *' from 'int' makes pointer from integer without a cast [-Werror=int-conversion]
   70 |     ptr = strptime(time_string, FMT, &timeptr);
      |         ^
cc1: all warnings being treated as errors

That's curious of course. It seems to be -Wall -Werror that's the problem. Check this:

$ make have_timegm   
cc -lm  have_timegm.o   -o have_timegm
$ ./have_timegm 'Thu Mar 17 01:44:38 2022 UTC'
seconds (0 - 60)	tm_sec: 38
minutes (0 - 59)	tm_min: 44
hours (0 - 23)		tm_hour: 1
day of month (1 - 31)	tm_mday: 17
month of year (0 - 11)	tm_mon: 2
year - 1900		tm_year: 122
day of week (Sunday=0)	tm_wday: 4
day of year (0 - 365)	tm_yday: 75
summer time		tm_isdst: 64
timestamp:		1647481478
success:		<Thu Mar 17 01:44:38 2022 UTC>

So it appears to work when compiled this way! But check this out. Compiling it like I described above where it just shows implicit functions when running it'll just segfault.

But what's truly fascinating there is the backtrace shows:

Core was generated by `./have_timegm Thu Mar 17 01:44:38 2022 UTC'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x00000000004012a0 in main (argc@entry=<optimized out>, argv@entry=<optimized out>) at have_timegm.c:76
76	    if (ptr[0] != '\0') {
Missing separate debuginfos, use: dnf debuginfo-install glibc-2.35-20.fc36.x86_64
(gdb) bt
#0  0x00000000004012a0 in main (argc@entry=<optimized out>, argv@entry=<optimized out>) at have_timegm.c:76
(gdb) p ptr
$1 = 0x6f5b5419 <error: Cannot access memory at address 0x6f5b5419>
(gdb) p *ptr
Cannot access memory at address 0x6f5b5419

So the return value of the function is messed up. I suppose this makes sense since it was implicitly defined so who knows what it actually ends up being. But still the bottom line is that this system IS modern and this should not be a problem.

bug report attachment

bug-report.20221106.015146.txt

Perhaps we should also have bug_report.sh compile run have_timegm.sh ? And if it fails we might want to see if the compilation is problematic like above so perhaps make it a bit different from the script in that stderr is not directed to /dev/null ? Just a thought.

Did you attach the above requested bug report file? [yes, no & why not]

Sure.

What is the airspeed Velocity of an Unladen Swallow? [n/a, :-), X m/s, unknown, etc.]

As amusing as this is I think we probably should remove it though because this will be confusing to some people. What do you think ?

Additional context

Perhaps some of the above could be here but it seemed relevant. Either way it seems that the feature test macro is not just a matter of older systems. This raises a question about what I cited from the Makefile: if this version of fedora also fails can we say that in 2024 we should be abandoning this? I think not.

Enhancement: discuss and resolve any remaining issues with `txzchk`

I'm opening this issue so that any issues that come up will not be lost in the other thread that's more OT stuff than anything else (or equally OT). I will copy paste any issues I'm aware of that I can easily access (which might depend on if Mail wants to load them properly as unfortunately GitHub makes it hard to find messages in long threads as they at different intervals don't load comments so have to scroll through long pages to find the right links .. I know what to search for I just don't know if I can get to them easily). Anyway that's what this issue is for.

If you would please assign it to me that would be great though maybe you'll want to assign it to you too since you discuss it as well: I leave that to you of course. One comment coming up shortly.

shellcheck fails under CentOS 7 for iocccsize-test.sh and mkiocccentry-test.sh

I'm letting you work this out because the iocccsize tool is special in that way and also I'm not even sure exactly what the expected output of the script should be so trying to fix it would be a problem.

Describe the bug

This is what you get under CentOS 7. Version info first:

ShellCheck - shell script analysis tool
version: 0.3.8

shellcheck: iocccsize-test.sh: hGetContents: invalid argument (invalid byte sequence)

Yes...it's incredibly helpful and in fact as helpful as watching dry paint dry but I tracked it down to this:

printf 'char str[] = "èéø";\r\n' >test-iocccsize/utf8.c
test_size utf8.c "12 21 1"

To Reproduce

Run shellcheck directly on iocccsize-test.sh under CentOS or run make shellcheck under CentOS (7).

Expected behavio(u)r

I think you know what should happen :)

In two commits today I fixed two scripts for shellcheck.

BTW apparently this was a bug that in shellcheck in 2015 (or a library that shellcheck uses? Not sure) but it's not been fixed or it's been fixed and either reintroduced or it wasn't completely fixed (or it's a related problem). Alternatively it's because CentOS 7 might not have a more recent update though 2015 seems a bit far behind to me: not sure.

tcc: error: invalid option -- '--pedantic'

-pedantic is a more standard option than --pedantic.

$ install/bin/tcc -O3 -g3 --pedantic
tcc: error: invalid option -- '--pedantic'
$ install/bin/tcc -O3 -g3 -pedantic
tcc: error: no input files

After changing "--pedantic" to "-pedantic", TCC successfully compiled the mkiocccentry.

Bug: mkiocccentry-test.sh fails - update after pull request #37 needed

As expected after pull request #37, the mkiocccentry-test.sh test now fails.

This issue was placed in note that there are modifications that are needed to finish pull request #37

On -a answers always overwriting answers

I think this is a mistake for the following reasons:

It no longer reminds the user how to use the answers file.
And maybe more importantly it means that a simple typo will overwrite the answers. This has happened to me more than once and although it's only test mode it could be annoying when the contest is running.

I suppose this is up to discussion but I believe the answers file should not be overwritten for these reasons.

Is there a reason you made it so it always overwrites the answers file?

Help wanted: compressed tarball sanity check tool: txzchk

Discussed in https://github.com/ioccc-src/mkiocccentry/discussions/39

^{Originally posted by lcn2 January 29, 2022}

We need help with more tools in this repo

You can help. See below.

We are working hard to be able to run a MOCK IOCCC in 2022

By MOCK IOCCC we mean a beta test run of an IOCCC where entries submitted would NOT be judged. Instead the IOCCC judges would go though the motions of running mock contest in order to beta test the process. Nobody would win the MOCK IOCCC, although we would certainly thank those who helped test the process with special thanks to those who found something in the MOCK process that we need to fix.

Any important lessons learned during the MOCK IOCCC would result in changes to the real IOCCC28.

Our goal is to hold the MOCK IOCCC in 2022 and then start the reason IOCCC28 in 2022.

There is A LOT of work that goes on behind the scenes by the IOCCC judges. You can help the IOCCC judges by helping us with the following tools related to mkiocccentry. See below.

JSON sanity check tools

Write a tool that attempts validate (performs sanity checks) on the .info.json produced by mkiocccentry.
Write a tool that attempts validate (performs sanity checks) on the .author.json produced by mkiocccentry

Those two tools could be written as pull requests to add them as separate programs to this repo. Moreover mkiocccentry could be modified to perform a function call into that code (in the same way the code iocccsize function is executed directly from mkiocccentry while iocccsize is also compiled as a separate tool).

Even though mkiocccenty would run the core of those tools as function calls, we still need the tools as standalone programs for use elsewhere. So both mkiocccentry callable functions AND standalone tools are needed.

The standalone tools should take a file as an argument. If sane, these tools should exit 0, otherwise exit non-zero.

compressed tarball filename sanity check tool

Write a tool that validates the filename of a compressed tarball file.

The mkiocccenty tool forms a compressed tarball file with a very specific form of a filename (for internal IOCCC judging process reasons). We need a tool that performs a sanity check on the filename of a compressed tarball file.

That tool could be added to this repo as a pull request. Moreover mkiocccentry could be modified to perform a function call into that code (in the same way the code iocccsize function is executed directly from mkiocccentry while iocccsize is also compiled as a separate tool) as a sanity check of what mkiocccentry produced.

Even though mkiocccenty would run the core of those tools as function calls, we still need the tool as standalone programs for use elsewhere. So both mkiocccentry callable functions AND standalone tool are needed.

The standalone tool should take a file as an argument. If sane, these tool should exit 0, otherwise exit non-zero.

compressed tarball sanity check tool

Write a tool to validate a compressed tarball produced by mkiocccentry