Change to launching monerode with --log-level 2 about archival_network

[Mitchellpkt]

@neptuneresearch @serhack Thoughts?

[Mitchellpkt]

Actually, --log-level 2 is extremely verbose... Maybe just 1

[serhack]

I agree with you. --log-level 1 is fine 👍

[Mitchellpkt]

Before we finalize this...

Perhaps we could run a daemon for 24 h at varying log levels and compare the space requirements.

Can we fill out this table:

*--log-level*      //     *daily_log_size*
   level 0                    ?? MB/day
   level 1                    ?? MB/day
   level 2                    ?? MB/day

[Mitchellpkt]

I can run this experiment, or let somebody else handle it. @serhack could we earmark one of the nodes for this kind of analysis for a few days (maybe Tokyo? Let me know if you have a preference or are working on one.)

[serhack]

The MAP-TOKYO-0 node could be OKAY. Before running the script, capture the disk size from Graphana. During the tests, MAP-TOKYO-0 can't install / update any program or manage any files besides the log file.

[neptuneresearch]

We could use the codebase to see what messages appear in each level.

Internally, these are calls to log functions by their level name. At a quick glance I see Debug, Info, Warning, Error, Fatal, Trace. So you could grep the codebase for all Trace calls for instance to see everything it puts under the Trace log level.

I could get together a list for you to cherry-pick. Or you could let me know your interested parameters
i.e. you need IP addresses, ... Or you can grep the codebase :)

The log messages could be a significant data source, and this way we would make a definite specification between the daemon log messaging and our analysis, working from the set of all messaging down to just what we want, instead of having to do multiple runs and working backwards from heterogeneous outputs.

FYI I have a 3 week logfile at log level 0 and its size is 16 MB.

[Mitchellpkt]

@serhack - nifty to watch from Graphana. I was going to compare the size of the log files themselves, but your method will let us see if anything else weird changes with that.

@neptuneresearch - I cannot find any good notes about the --log-level flag or its various levels. That sounds like a good side project, if you want to poke at the codebase and assemble some documentation. Perhapse we could start collecting that info in the wiki??

It would be good to see how --log-level 1 versus --log-level 2 record the following events:

• incoming transactiosn
• incoming blocks
• alternative blocks
• reorgs

[Mitchellpkt]

Thanks @neptuneresearch - adding 16 MB / 21 d ~ 0.75 MB/day to the table

*--log-level*      //        *daily_log_size*
   level 0                    0.75 MB/day (NR)
   level 1                    ?? MB/day
   level 2                    ?? MB/day

[neptuneresearch]

Ok I can make this list.

Not sure about in 2 days though, so don't wait to get any logs going in the meantime. I can tell you more about any specific log messages, as well. I'd just go look at the code surrounding them.

[Mitchellpkt]

Cool. @neptuneresearch - 2 days??? This is very low priority.

I'm going to do the logsize tests, and jot down the events listed above.

I'll write up notes based on that, then pass them off to you (well, the wiki) and you can add anything else that you find in the code.

Definitely no rush.

[Mitchellpkt]

Looks like logs compress pretty well...

$ ls -la
-rw-r--r--  1 m m   4022961 Jul 27 11:42 bitmonero-TOKYO-1day.7z
-rw-r--r--  1 m m 104850022 Jul 27 11:38 bitmonero-TOKYO-1day.log
-rw-r--r--  1 m m   4023824 Jul 27 11:42 bitmonero-TOKYO-1day.tar.xz
-rw-r--r--  1 m m   6801306 Jul 27 11:41 bitmonero-TOKYO-1day.zip

We could have a compressor script that keeps an eye on ~/.bitmonero/, and whenever it sees a <filename> that matches bitmonero.log-*. it executes

tar -cf <filename>.tar <filename>
xz -z <filename>.tar
rm <filename>

[neptuneresearch]

Oh, please note the 16 MB figure was from the archive daemon which additionally logs:

• 1 line per block
• the alt_chain_info every block, this can be several lines long

So you will not see this extra size on the stock daemon.

I was actually just changing around the archive logging. In version 6 I currently have:

• Block message:

  Block Archive [Chain=MAIN/ALT H=00000000 MRT=00000000 NRT=00000000]

Note: the 00.. stands for a number, the length may not be accurate in this example.

• Altchaininfo:
  I'm taking this out, this is duplicate to the saved altchaininfo data so this was really just a convenience for watching it from the daemon console.

Comments on the Block message? Anything else you would like to see in it?

[neptuneresearch]

For the record, it looks like the archive daemon logs archive messages under the Info level to the global log.

[Mitchellpkt]

That looks good. I am extremely eager to have MRT/NRT information.

If it won't hold up the patch timeline, then it would be natural to record peer information alongside the NRT for each copy. However if that would set you back more than an hour or two, just save it for the following version of the patch. Since we're totally blind at the moment, [patch without peer info, soon] >> [patch with peer info, later]

[neptuneresearch]

Yea, no peer info, I haven't researched that yet. Probably when I do the log messaging analysis.

[Mitchellpkt]

Realized that --log-level 4 contains data we've been looking for (txn_hash + txn_NRT for first copy), so we can get more mileage out of the logging than we realized.

It would be cleaner to set custom categories rather than overkill --log-level 4 with extraction. Notes from stoffu in MRL about acomplishing this:

Stoffu Noether > See monero-project/monero@5833d66 for a detailed explanation.

For example, you can set --log-level 0,*net*:DEBUG to specifically make any log categories containing net such as net.p2p print debug level logs.

Since I'm tied up with Mastering Monero right now, I just let a new node sync from scratch with monerod-archive and --log-level 4 ... Once I finish a few more chapters, I'll dig through that those logfiles to document desired information and specification by input flag.

[neptuneresearch]

Edit: I moved this information to the Wiki at Monerod Log Levels.

IMO, *:TRACE (log level > 3) is a lot, maybe we could lower the noise-to-signal ratio if we only :TRACE what we need and otherwise use *:WARNING.

[neptuneresearch]

@IsthmusCrypto just to touch base on this, we are log level 1 currently until you go through the above doc and pick out more categories [if] you want.