Custom properties don't get submitted to ElasticSearch after upgrading to 2.0.0 (from 1.1.0) about log4stash

[urielha]

Very weird, what you're saying is that the message is getting into Elastic but without any custom values?
Do you see any error message or any other indication?

Because I have added a specific test for custom values and I won't submit this version if the test wasn't pass...

I hope I could test this soon with your config.
Please update if you have more insights.

Regards

[urielha]

I can't repro this on my local machine.
Do you still get the count: 0 in you json or you get something else now?

Please check again that you are using the correct version and see if you can find any error message.

[jonhb]

I can't get custom properties to work either. I set them with GlobalContext.Properties["Guid"] and use %property{Guid} to access them. This works fine in fileappender but not in ElasticSearchAppender.

I get this in ES: "Guid": {}

Do I have to set something up in the config for these custom properties?
All I have is this:

    <appender name="ElasticSearchAppender" type="log4stash.ElasticSearchAppender, log4stash">
      <Server>localhost</Server>
      <Port>9200</Port>
      <IndexName>logstash-%{+yyyyMM}</IndexName>
      <IndexType>log</IndexType>
      <IndexAsync>true</IndexAsync>
    </appender>

[urielha]

Hi @jonhb and @Dima234 ,
Please try this pre release:
https://www.nuget.org/packages/log4stash/2.0.2-pre

Let me know if it fixed your issue.
Thanks

[jonhb]

Hi,

I still can't get it to work.

[urielha]

emmm... let's try another thing,
try to put:

  <FixedFields>Partial</FixedFields>

inside your configuration file.
(so it will be:

    <appender name="ElasticSearchAppender" type="log4stash.ElasticSearchAppender, log4stash">
        <Server>localhost</Server>
        <Port>9200</Port>
      <IndexName>logstash-%{+yyyyMM}</IndexName>
      <IndexType>log</IndexType>
      <IndexAsync>true</IndexAsync>

      <FixedFields>Partial</FixedFields>

    </appender>

And furthermore, I noticed that you said you

use %property{Guid} to access them

Where do you use it?

Hope the FixedFields will make it work.

Cheers

[jonhb]

Hi,

I use it where I define the output pattern for the RollingFileAppender (and also where i define parameters for the SQL server appender):

`

`

This outputs the Guid correctly in the file. In the code I set the Guid through a provider class where I have overridden the ToString method to return the Guid variable.
This is only initialized once:
log4net.GlobalContext.Properties["Guid"] = new GuidProvider();
and then this returns the dynamic Guid:
public class GuidProvider { public override string ToString() { return Guid; } }

The FixedFields does not help anything unfortunatly.

[urielha]

Thanks, I'm pretty sure I found the problem.

Will update you soon with new version.

Note that your problem is different from the author of this thread that is why I got a bit confused at first.

[urielha]

Forgot to ask, Is there a reason you don't call toString inline?
like this:

log4net.GlobalContext.Properties["Guid"] = new GuidProvider().toString();

[jonhb]

I don't think that will work.

[urielha]

Please use this version

https://www.nuget.org/packages/log4stash/2.0.2-b

@jonhb, Add this to your configuration :

<ElasticFilters>

  <Convert>
    <ToString>Guid</ToString>
  </Convert>

</ElasticFilters>

This will cause calling the ToString() function before indexing into ElasticSearch.

@Dima234, Add this to your configuration:

<ElasticFilters>

 <Convert>
    <ToString>UserName</ToString>
    <ToArray>
      <SourceKey>UserName</SourceKey>
    </ToArray>
  </Convert>

</ElasticFilters>

This will cause calling ToString() of the stack object and then convert the values to array.

If you both already have ElasticFilters just add the Convert into it.
And please open new issue if the problem still occurs.

All the best,
Uriel