Facing `org.apache.hadoop.ipc.RpcNoSuchMethodException` when connecting to hdfs using nameservice about hdfs-native

[Kimahriman]

Also, could you please help me understand why the msync check is done, if the error is eventually silenced.

It's for newer HDFS clusters that use Observer NameNodes. The Java client does an msync on startup so that it knows what the latest transaction ID in a cluster is. It makes sure the first read for a new client is never stale. I need to rework some of the handling for Observer NameNodes still, but wanted to keep that in there for now (my team uses observer nodes)

[Kimahriman]

I eagerly do an msync which is a newer hdfs thing. I catch if it's not supported but not if the method doesn't exist for older hdfs versions. In general I'm not trying to be compatible with older hdfs versions (2.6 is very old), but this one I can add a check for and just ignore the failure

[shbhmrzd]

Thank you so much!
I have added method not found in the ignore list as well.
Please take a look at the change request - #55

Also, could you please help me understand why the msync check is done, if the error is eventually silenced.

[shbhmrzd]

Thank you!
It does not fail anymore.

But the run gets stuck. Have you seen this issue before or can think of any possible cause?

[shbhmrzd]

Found the issue, list_status is passed recursive true which leads to this(there are multiple sub dirs). Worked fine with false.
Thank you!