irods / irods_client_icommands Goto Github PK

CMake 1.77% Shell 1.30% C++ 72.16% Lex 0.29% Yacc 1.62% R 9.96% Python 12.77% Rez 0.13%

irods_client_icommands's Introduction

iRODS Client - iCommands

This repository hosts the client iCommands, the default command line interface to iRODS.

The client iCommands are released as a single package called irods-icommands in the repository at https://packages.irods.org/

Once the repository is configured, the irods-icommands package can be installed via the package manager on any supported OS.

Build

To build the iCommands, you will need the irods-dev and irods-runtime packages.

This is a CMake project and can be built with:

cd irods_client_icommands
mkdir build
cd build
cmake -GNinja ../
ninja package

Install

The packages produced by CMake will install the ~50 iCommands, by default, into /usr/bin.

Build without any Package Repositories

If you need to build the iRODS iCommands without the use of any APT/YUM repositories, it will be necessary to build all the dependencies yourself. The steps include:

Download, build, and install packages from https://github.com/irods/externals
Update your PATH to include the newly built CMake
Download, build, and install irods-dev(el) and irods-runtime from https://github.com/irods/irods
Download, build, and install irods-icommands from https://github.com/irods/irods_client_icommands

Our dependency chain will shorten as older distributions age out.

The current setup supports new C++ features on those older distributions.

Userspace Packaging

A userspace-tarball buildsystem target is provided to generate a userspace tarball package. This package will contain the iCommands and all required library dependencies. See packaging/userspace/build_and_package.example.sh for an example of how to build and package the iCommands for userspace deployment.

Build Dependencies

The userspace packager needs a few extra packages to work properly:

Required: Python 3.6+.
Required: setuptools Python 3 package.
- Available as python3-setuptools via yum/apt on Centos 7/Ubuntu.
- Available as setuptools on PyPI.
Recommended: distro Python 3 module.
- Required for Python 3.8+.
- Available as python3-distro via apt on Ubuntu 18.04+.
- Available as python36-distro via yum on Centos 7.
- Available as distro on PyPI.
Recommended: lief Python 3 module, version 0.10.0+ (preferably 0.11.0+).
- Available as lief on PyPI.
Recommended: chrpath tool
- Available as chrpath via yum/apt on Centos/Ubuntu.

If you've got pip, the following one-liner should get all the Python dependencies installed:

python3 -m pip install lief setuptools distro

irods_client_icommands's People

Contributors

Stargazers

Watchers

irods_client_icommands's Issues

iadmin can only change passwords when connected to a catalog provider server

Currently, iadmin moduser makes use of the challenge string provided by the server and the ObfEncodeByKeyV2 function to encode passwords. This challenge string is generated on a per-server basis, so when the password request change is initiated forwarded from a resource server to a catalog server to actually decode, scramble, and store the new password in the database, the decode fails.

This would be an acceptable limitation if the interface required it. However, the generalAdmin interface passes what it's sent on to the database plugin in the exact same way that the interface ipasswd uses (userAdmin) does, and ipasswd does not make use of or require a challenge, and, indeed, works when connected to a catalog consumer server.

This is because the database plugin's mod user function will decode passwords WITH OR WITHOUT a challenge encoded in them, based on whether or not the challenge was encoded. Therefore, we can just encode passwords for iadmin moduser in the same way we do for ipasswd. Because they are still encoded using a known password, they are still secure to the same degree any API call made by ipasswd is.

Install icommands on CentOS 8 (arch: x84_64) and Red Hat 8 (arch: ppc64le)

Dear all,

we need to install icommands on the login nodes of our HPC clusters:
The login nodes have these OS/architectures:

Intel Xeon Platinum 8276-8276L (x84_64) with CentOS 8.3
IBM Power9 AC922 (ppc64le) with Red Hat 8.1

We have already seen that there are no rpm packages available for CentOS 8 and RedHat 8.

On x86_64 I have already tried to install icommands inside a singularity container with CentOS 7 and they seem to work correctly. Is there any known issue with using icommands for CentOS 7 on CentOS 8? Is there another way to install icommands on CentOS 8 without containers?

On ppc64le I wasn't able to build a container with I icommands, there is no rpm available for ppc64le/centos7. Do you have any suggestions?

Best,
Francesco

iquest reads query from stdin

This is a feature request.

When scripting with iquest in bash, it would be nice to be able to use iquest with heredocs. This allows complex queries to span multiple lines without having to add a lot of escape characters. This improves readability. For example.

iquest --no-page '%s' <<QUERY
select DATA_RESC_HIER
where DATA_RESC_NAME = 'CyVerseRes' 
   and COLL_NAME = '$coll' || like '$coll/%'
   and DATA_NAME lile '%.csv'
QUERY

instead of

iquest --no-page '%s' \
   "select DATA_RESC_HIER where DATA_RESC_NAME = 'CyVerseRes' and COLL_NAME = '$coll' || like '$coll/%' and DATA_NAME lile '%.csv'

iquest parses queries with tabs in the where clause incorrectly.

I'm using iRODS iCommands 4.2.8.

Where there is a horizontal tab in the where clause of a generalized query, iquest doesn't return any results. In the following example there is a horizontal tab immediately before and,

prompt> printf -v query $'select DATA_NAME, DATA_SIZE where COLL_NAME = \'/iplant/home/tedgin\'\tand DATA_NAME like \'test%%\''
prompt> iquest '%s %s' "$query"
CAT_NO_ROWS_FOUND: Nothing was found matching your query

If the horizontal tab is replaced by a space, iquest works as expected.

prompt> printf -v query $'select DATA_NAME, DATA_SIZE where COLL_NAME = \'/iplant/home/tedgin\' and DATA_NAME like \'test%%\''
prompt> iquest '%s %s' "$query"
test.tgz 110084063
test0B 0
test100GiB 107374182400

Make iRODS commands RPM package relocatable

Could the iRODS icommands package be configured to support the relocatable option in the RPM package?

This describes the option:
http://www.rpm.org/max-rpm/ch-rpm-reloc.html

iinit does not document accepting a password as a command line argument

iinit's help text needs to be updated to warn users that the password will be in clear-text.

Housekeeping (typos, grammar, whitespace)

better verbosity for writes

We're using icommands & server both 4.2.7 on Ubuntu Bionic.

The writing commands we (we locally being the /cgp zone) use are

iput of "small" files: doesn't tell the destination
iput of "big" files: -V flag will tell destination server, but not the resource
irepl, any size: doesn't tell the destination

I'm asking specifically about write commands because we're seeing some odd performance problems which @kript and @bh9 are dealing with for us, but I can't so easily generate stats on "what transfers were fast, and which were slow?" when the icommands don't say where the file was put.

Things that would be useful,

tell the DATA_ID of the resulting object
tell the DATA_RESC_ID(s) of resulting copies as they're created
machine readable output
consistent output format & contents, regardless of the different code paths used to do the work
consistent output format, even when the operation fails
efficiency; using information already to hand, not spending time or generating load by doing more lookups.

I realise this isn't going to happen in time to deal with our current problem, but please can it guide future changes to icommand output?

Without the above I may look at the before & after of irepl happening. One significant advantage of this in the current contrext is that I have "before" already, with the DATA_ID of these files in bulk. From that I can discover later where these files ended up, and it's much more efficient to do them in bulk:

$ iquest '%s %s:%s %s/%s' "select order(DATA_ID), order(DATA_REPL_NUM), DATA_RESC_ID, COLL_NAME, DATA_NAME where DATA_ID in ('49188939', '49189577', '49189592', '49189595', '49201390', '49201414', '49201434', '49203573')"
49188939 0:48662204 /cgp/intproj/2500/sample/foo/foo.v1.sample.dupmarked.bam
49188939 1:48624007 /cgp/intproj/2500/sample/foo/foo.v1.sample.dupmarked.bam
49189577 0:38224944 /cgp/intproj/2500/sample/foo/foo.v1.sample.dupmarked.bam.bai
49189577 1:41692990 /cgp/intproj/2500/sample/foo/foo.v1.sample.dupmarked.bam.bai
49189592 0:48662204 /cgp/intproj/2500/sample/foo/foo.v1.sample.dupmarked.bam.met.gz
49189592 1:41692989 /cgp/intproj/2500/sample/foo/foo.v1.sample.dupmarked.bam.met.gz
49189595 0:41704469 /cgp/intproj/2500/sample/foo/foo.v1.sample.dupmarked.bam.bas
49189595 1:41693077 /cgp/intproj/2500/sample/foo/foo.v1.sample.dupmarked.bam.bas
49201390 0:38224943 /cgp/intproj/2500/sample/foo/foo.v1.ascat.counts.gz
49201390 1:41692991 /cgp/intproj/2500/sample/foo/foo.v1.ascat.counts.gz
49201414 0:48662203 /cgp/intproj/2500/sample/foo/foo.v1.ascat.counts.gz.tbi
49201414 1:48624007 /cgp/intproj/2500/sample/foo/foo.v1.ascat.counts.gz.tbi
49201434 0:41704467 /cgp/intproj/2500/sample/foo/foo.v1.ascat.counts.is_male.txt
49201434 1:41693079 /cgp/intproj/2500/sample/foo/foo.v1.ascat.counts.is_male.txt
49203573 0:48662208 /cgp/intproj/2500/sample/foo/foo.v1.merged.bw
49203573 1:41692984 /cgp/intproj/2500/sample/foo/foo.v1.merged.bw

This way I have the option of patching over the lack of verbosity with more programming but less runtime, because I can query around 1000 individual files or a range of 100k files in one bite.

Current outputs look like this, when I ask for Christmas-tree verbosity:

$ iput -v -V -P -f MANIFEST /cgp/sandbox/mca/MANIFEST
0/1 -  0.00% of files done   0.000/0.003 MB -  0.00% of file sizes done
Processing MANIFEST - 0.003 MB   2021-07-01.11:27:25
   MANIFEST                        0.003 MB | 0.531 sec | 0 thr |  0.005 MB/s

$ iput -v -V -P -f ~/ln.any-old.bam /cgp/sandbox/mca/any-old.bam
0/1 -  0.00% of files done   0.000/12531.713 MB -  0.00% of file sizes done
Processing ln.any-old.bam - 12531.713 MB   2021-07-01.11:05:07
From server: NumThreads=16, addr:irods-cgp-sb01, port:20185, cookie=48059554
ln.any-old.bam - 280.000/12531.713 MB -  2.23% done   2021-07-01.11:05:12
ln.any-old.bam - 720.000/12531.713 MB -  5.75% done   2021-07-01.11:05:12
ln.any-old.bam - 1320.000/12531.713 MB - 10.53% done   2021-07-01.11:05:12
ln.any-old.bam - 1840.000/12531.713 MB - 14.68% done   2021-07-01.11:05:13
ln.any-old.bam - 2520.000/12531.713 MB - 20.11% done   2021-07-01.11:05:13
ln.any-old.bam - 3040.000/12531.713 MB - 24.26% done   2021-07-01.11:05:14
ln.any-old.bam - 3400.000/12531.713 MB - 27.13% done   2021-07-01.11:05:14
ln.any-old.bam - 4120.000/12531.713 MB - 32.88% done   2021-07-01.11:05:14
ln.any-old.bam - 4840.000/12531.713 MB - 38.62% done   2021-07-01.11:05:15
ln.any-old.bam - 5400.000/12531.713 MB - 43.09% done   2021-07-01.11:05:16
ln.any-old.bam - 6160.000/12531.713 MB - 49.16% done   2021-07-01.11:05:16
ln.any-old.bam - 6520.000/12531.713 MB - 52.03% done   2021-07-01.11:05:17
ln.any-old.bam - 7240.000/12531.713 MB - 57.77% done   2021-07-01.11:05:17
ln.any-old.bam - 8040.000/12531.713 MB - 64.16% done   2021-07-01.11:05:18
ln.any-old.bam - 8640.000/12531.713 MB - 68.95% done   2021-07-01.11:05:18
ln.any-old.bam - 9440.000/12531.713 MB - 75.33% done   2021-07-01.11:05:19
ln.any-old.bam - 9880.000/12531.713 MB - 78.84% done   2021-07-01.11:05:19
ln.any-old.bam - 11063.232/12531.713 MB - 88.28% done   2021-07-01.11:05:20
ln.any-old.bam - 11836.160/12531.713 MB - 94.45% done   2021-07-01.11:05:21
ln.any-old.bam - 12095.553/12531.713 MB - 96.52% done   2021-07-01.11:05:21
ln.any-old.bam - 12531.713/12531.713 MB - 100.00% done   2021-07-01.11:05:21
   any-old.bam                 12531.713 MB | 37.172 sec | 16 thr | 337.124 MB/s
$ irepl -R red -v -V -P /cgp/sandbox/mca/any-old.bam
0/1 -  0.00% of files done   0.000/12531.713 MB -  0.00% of file sizes done

so I had since 3.3.1 regarded the -v and -V flags as producing no useful information, and avoided using them.

Release 4.2.0 ?

As irods 4.2.0 has been released, could you also release irods_client_icommands 4.2.0 ?

irsync -r stops in a directory where an error occurs

I created a directory (test) with files:

file1.txt, file2.txt, file3.txt, file4.txt

In addition to these, I created a symbolic link dne.txt that point to a non-existent file.

When I run irsync -rv /home/jjames/test i:test, I get a USER_INPUT_PATH_ERR due to the invalid symbolic link. That is good. However, the other files within /home/jjames/test do not get synced. Sometimes some of the files will be synced but not all. In this case none of them did.

Note: If this is recursive and there are other higher level directories being synced, the irsync will continue on those. If it is to continue elsewhere it should continue with all files in the current directory as well.

some rule file tests failing

4-2-stable
main

Fix failing test(s) in icommands rule file test suite.

The following are related changes that allow these tests to run (spec. on main) and accurately reflect the results of successful / failed rule runs as test passes and fails respectively.

irods/irods#6206
irods/irods#6170

Unable to read irule input arguments in rule

Since I upgrated to 4.2.0, I am facing following error:

The input line in the .r rule file specifies arguments as following
INPUT *filePath=$, *fileSize=$, *mainResource=$
and the rule is called according to irule's help page:
irule -s -F customPut.r *filePath=/my/path *fileSize=3125 *mainResource=myResc

Still that, supplied values are not available during execution

status = -1211000 RE_UNABLE_TO_READ_LOCAL_VAR
Level 0: DEBUG: error: unable to read local variable *filePath
line 2, col 20

irods-icommands installs without irods-runtime on CentOS 8

I was able to install version 4.3.0 of iCommands on a CentOS 8 machine by following the instructions on https://packages.irods.org and doing yum install irods-icommands. Unfortunately, the dependency irods-runtime wasn't automatically installed, and I had to install it with a second yum call, yum install irods-runtime.

imeta aborts with an uncaught exception on commands when the file is not provided.

In many places imeta detects an error due to missing arguments but does not return immediately which leads to uncaught exceptions. The code should return with SYS_INVALID_INPUT in these cases.

Examples:

imeta ls -d
imeta ls -C
imeta mod -d
imeta mod -d file.dat
imeta rm -d
etc.

This is related to #18 but is meant to be more general to cover all of these cases.

icp cannot copy files larger than 2GB

Hi,

I'm using icommands 4.3.0.

When copying a file larger than 2GB using icp fails with following error.

ubuntu@irodsfs-test:/data$ icp /iplant/home/iychoi/test_2GB.bin /iplant/home/iychoi/tt/test_2GB.bin
remote addresses: 206.207.252.46 ERROR: cpUtil: cp error for /iplant/home/iychoi/tt/test_2GB.bin, status = -27000 status = -27000 SYS_COPY_LEN_ERR

However, iput worked correctly.

The file I tried has 2147483648 bytes which is 1 byte larger than int max (2147483647). So I guess signed int32 data type was used in offset/size calculation in the iRODS API for copying. This is just my guess.

Thanks,
Illyoung Choi

Provide binary (non-packaged) distribution of icommands

I'm trying to use the icommands in an HPC environment in which I have no admin privileges, which means I can't use .deb and .rpm distributions. Additionally, building the icommands requires a lot of build tools which I also can't install for the same reasons. Could you provide a binary distribution of the icommands where you just provide the binaries and any dynamic libraries that are necessary, in addition to the .deb and .rpm packages?

imeta rm uncaught exception if too few arguments

$ iput VERSION.json
$ imeta rm -d VERSION.json a

Error: Not enough arguments provided to rm

terminating with uncaught exception of type boost::exception_detail::clone_impl<boost::exception_detail::error_info_injector<boost::bad_any_cast> >: boost::bad_any_cast: failed conversion using boost::any_cast
Aborted (core dumped)

The existence of the "value" argument is checked for here, but it is then unconditionally used here.

After brief inspection, should probably return early with something like SYS_INVALID_INPUT_PARAM.

This error is repeated throughout the file.

Inconsistent iinit irods_authentication_scheme setting for pam in 4.3.0

When using the pam(_password) method for authentication in irods, it is expected that .irods/irods_environment.json contains a value for irods_authentication_scheme that indicates pam is needed for logging in.

Up to version 4.2.11, the value that should be specified, was "PAM". In 4.3.0 this apparently has changed to "pam_password" (see #314), leading us to include a line

iinit -h | grep Version | grep -v -q 4.3.0 || sed -i 's/"irods_authentication_scheme": "PAM"/"irods_authentication_scheme": "pam_password"/' ~/.irods/irods_environment.json

in the snippet we distribute to our users for authentication.

However, as it turns out, the deb release of 4.3.0 for focal https://packages.irods.org/apt/, still expects "PAM", while the rpm release of 4.3.0 on https://packages.irods.org/yum/ now expects "pam_password". I.e. the above version check is not enough.

irsync --link fails on broken symlink

We are facing a similar problem as reported in the closed issue #22.
When we use irsync with the option --link, soft links are ignored except when they are broken.

What did you try to do?

A user wants to upload a directory, which contains (broken) symlinks.
I wanted to check whether we could exclude symlinks with the --link option

What did you expect to happen?

When you upload a symlink with irsync --link i:, it is ignored, no matter whether the symlink works or not.

The main reason why some users were interested in this option in the past is because they have potentially broken symlinks, and would like to avoid errors when syncing large collections.

Note: I'm a bit confused whether this is the intended behaviour or not.
Based on #22, a broken symlink should be ignored, not throwing an error.
However, the thread which it refers to as solution (irods/irods#3988) seems to imply the error is intended behaviour.

What was the actual result?

Working symlinks are ignored, but broken symlinks throw the following error:

remote addresses: 172.23.253.137 ERROR: resolveRodsTarget: source brokenlink does not exist
remote addresses: 172.23.253.137 ERROR: rsyncUtil: resolveRodsTarget status = -317000 USER_INPUT_PATH_ERR

If you upload a directory with a broken symlink in it, the error is as follows:

Running recursive pre-scan... remote addresses: 172.23.253.137 ERROR: USER_INPUT_PATH_ERR: No such file or directory
Path = test_symlinks/brokenlink

pre-scan complete... errors found.
Aborting data transfer.

This causes other files in the directory to not be synced.

Minimal reproducable example

# create two files
echo "hello" > reference1
echo "hello" > reference2

# create soft links
ln -s reference1 workinglink
ln -s reference2 brokenlink

# remove the second file, creating a broken link
rm reference2

# execute irsync --link on good link
# nothing happens, no error occurs
irsync --link workinglink i:workinglink

# execute irsync --link on good link
# will throw a USER_INPUT_PATH_ERR error
irsync --link brokenlink i:brokenlink

Environment details

I tested this on the following setups:

iRODS 4.3.1 with iCommands 4.3.1 and a Mysql database (production environment)
iRODS 4.3.1 with iCommands 4.3.1 and a Mysql database (test environment)
iRODS 4.3.1 with iCommands 4.3.1 and postgres database (test environment)

To be clear, I'm not intending on syncing anything in the holiday period :)
But I wanted to report it before losing track of it.
If I have anything wrong, just let me know.

Best regards and happy holidays,
Jef

Add iclienthints command to get client hints

iCommands 4.3.0 fails to run with PAM

Hello, i've installed icommands on multiple machines and version 4.3.0 fails to run with pam module.

Error occurred while authenticating user [dvo0012] [PLUGIN_ERROR_MISSING_SHARED_OBJECT: [-]     /usr/include/irods/irods_load_plugin.hpp:157:irods::error irods::load_plugin(PluginType *&, const std::string &, const std::string &, const std::string &, const Ts &...) [PluginType = irods::experimental::auth::authentication_base, Ts = <char [14]>] :  status [PLUGIN_ERROR_MISSING_SHARED_OBJECT]  errno [] -- message [shared library does not exist [/usr/lib/irods/plugins/auth/libirods_auth_plugin-pam_client.so]]



] [ec=-1827000] failed with error -1827000 PLUGIN_ERROR_MISSING_SHARED_OBJECT

icommands 4.2.7 works correctly. Please can you look at it?

Usage of `get_grid_configuration` and `set_grid_configuration` not clear

We would like to test 4.3.1. And for this, we have a test environment running irods 4.3.1, iCommands 4.3.1(main) on almalinux8.

I wanted to test sub-commands/features done through the commits available for irods/irods#7274. Also I am seeing some documents https://docs.irods.org/4.3.1/system_overview/configuration/#configuring-authentication-in-r_grid_configuration and here https://github.com/irods/irods_client_icommands/blob/main/src/iadmin.cpp#L2672-L2692. However, I dont see how to use these subcommands in iadmin -h, they seem not exist.

But if I run these sub commands, I am getting:

[irods@dev-u0137480 irods]$ iadmin get_grid_configuration authentication password_max_time
Failed to get grid configuration for namespace [authentication] and option [password_max_time] [ec=-808000]

Also R_GRID_CONFIGURATION table (alanking/irods_docs@21fbfd0#diff-36178197e1823e07e5432ebea993d7e94af368f2b7160359ee4c61b3f8cddcc0R890) doesnt have more than:

MySQL [irods]> select * from R_GRID_CONFIGURATION;
+--------------+----------------+---------------------------------------+
| namespace    | option_name    | option_value                          |
+--------------+----------------+---------------------------------------+
| database     | schema_version | 11                                    |
| delay_server | leader         | dev-u0137480.irods.t.icts.kuleuven.be |
| delay_server | successor      |                                       |
+--------------+----------------+---------------------------------------+
3 rows in set (0.001 sec)

This repo doesnt have 4.3.1 release for iCommands yet. Am I missing something?

ihelp does not mention iunreg

In version 4.2.8, of iCommands, ihelp doesn't mention the new iCommand iunreg.

problem irm of a file in nested directory

Dear all,
when I try to remove a file in a nested directory with a very long tree name, I obtain the error

ERROR: main: parseCmdLinePath error. status = -346000 USER_PATH_EXCEEDS_MAX

I'm using icommand version 4.2.0-1
How can I solve?

Best regards. Giusy

iput numThreads

Can I use multi threads when I run iput?

I want to change the number of threads during iput execution, what should I do?

iput -N 32 data

Is this the right way to use it?

I wonder if you currently support multi-threads.

showing metadata ID

Due to this bug irods-contrib/metalnx-web#234 I got some AVU entries in ICAT which I was not able to delete with imeta rm ..., iRODS complained about empty string being invalid, I was able to delete it with "imeta rmi...", but I do not see an obvious way to obtain the metadata ID with the icommands (I got that info with a little python-irodsclient script).

I suggest to print the metadata ID with imeta ls -V...

By the way, imeta ls -d file, imeta ls -d -v file and imeta ls -d -V file print the exact same output in my test environment which leaves me somewhat confused about the point of the options -v and -V.

Building icommands on i386 machine

Dear all,

I am currently in the process of building the irods-icommands package on our i386 machines, we currently have around 640 i386 machine in our web archive cluster. I was wondering where I can find the irods-dev and irods-runtime source code to build the packages on our i386 systems. I have checked the irods repository and found the i386 packages empty.

Thank you in advance for your help.

iquest silent when no results

This is a feature request.

It would be nice if iquest were silent when the query yields no results. At the moment, it responds with CAT_NO_ROWS_FOUND: Nothing was found matching your query/ This makes it hard to script with iquest. When iterating over a result set, this value needs to be filtered out to prevent the script from failing in a weird way when the result set is empty.

CAT_NO_ROWS_FOUND: ... looks like an error, but the exit status of iquest indicates success, which is a little confusing.

Maybe iquest could have two modes. The first mode would treat an empty result set as an error. iquest would write CAT_NO_ROWS_FOUND: ... to stderr and exit with a failure status. The second mode would not treat an empty result set as an error. It would write nothing to stderr or stdout and would exit with a success. The first mode could be the default. The mode could be selected by passing a command line option --empty=(failure|success) something like that. If no option is provided the default could be the first mode.

build error in rocky with main branch

hi @korydraughn
after finished in irods server
I met another problem,
error message showed as below, is that matter with something update? or it just not finished yet.
Thank you

error message:
[ 73%] Building CXX object CMakeFiles/imeta.dir/src/imeta.cpp.o
/root/irods_client_icommands/src/imeta.cpp:1000:38: error: no member named 'condInput' in 'modAVUMetadataInp_t'
addKeyVal(&modAVUMetadataInp.condInput, ADMIN_KW, "");
~~~~~~~~~~~~~~~~~ ^
/root/irods_client_icommands/src/imeta.cpp:1093:38: error: no member named 'condInput' in 'modAVUMetadataInp_t'
addKeyVal(&modAVUMetadataInp.condInput, ADMIN_KW, "");
~~~~~~~~~~~~~~~~~ ^
2 errors generated.
make[2]: *** [CMakeFiles/imeta.dir/build.make:76: CMakeFiles/imeta.dir/src/imeta.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:2191: CMakeFiles/imeta.dir/all] Error 2
make: *** [Makefile:156: all] Error 2

istream needs to print the rError stack

master
4-2-stable

Enable C++17 Support

Allow multiple icommands versions to be installed

We often find ourselves in the situation where we run one version of iRODS in production, and another in test. Currently only one version of the icommands can be installed at a time, and since run-in-place is no longer supported or production, we can't build them ourselves and place in, say /opt/renci/4.2.7 etc...

It would be very helpful if the icommands could be installed in a way that is version specific, e.g. /usr/bin/icommands_4.1.12/ils, /usr/bin/icommands_4.2.7/ils (or even preferably /opt/renci/4.2.7/ )etc, and have a way in one's irods_environemnt.json file to set the version required (with, say, the latest being the default).

This is probably related to #66

Thank you!

add man pages to 4.2 packaging

crib logic from https://github.com/irods/irods/blob/4-1-stable/packaging/build.sh#L1555

usr directory error

Hi, glad to meet you again.
I followed (this) [https://docs.irods.org/4.2.3/getting_started/installation/#non-package-install-run-in-place] documentation, in 'Non-Package Install (Run In Place)' section.

user@hostname:~ /irods/ $ export IRODS_INSTALL_DIR=/path/to/the/non-package-root
user@hostname:~ /irods/ $ mkdir build
user@hostname:~ /irods/build $ cmake -DCMAKE_INSTALL_PREFIX=$IRODS_INSTALL_DIR -DIRODS_EXTERNALS_PACKAGE_ROOT=$IRODS_EXTERNALS ../
user@hostname:~ /irods/build $ make non-package-install-postgres

I've done it up to here. But There is no /usr in the $IRODS_INSTALL_DIR.

user@hostname:~ /irods_client_icommands/build $ cmake -DCMAKE_INSTALL_PREFIX=$IRODS_INSTALL_DIR -DIRODS_DIR=$IRODS_INSTALL_DIR/usr/lib/irods/cmake ../
user@hostname:~ /irods_client_icommands/build $ make install
user@hostname:~ /irods_client_icommands/build $ export LD_LIBRARY_PATH=$IRODS_INSTALL_DIR/usr/lib
user@hostname:~ /irods_client_icommands/build $ export PATH=$IRODS_INSTALL_DIR/usr/bin:$IRODS_INSTALL_DIR/usr/sbin:$PATH

So, /usr was removed and the execution was completed.

user@hostname:/var/lib/irods $ python ./scripts/setup_irods.py

However, an error occurs here.
Here is my error message.

syslab@syslab:~/workspace_chc/install_irods/var/lib/irods$ sudo python ./scripts/setup_irods.py
[sudo] password for syslab:
Warning: Hostname `syslab` should be a fully qualified domain name.
Error encountered running setup_irods:
Traceback (most recent call last):
  File "./scripts/setup_irods.py", line 447, in main
    test_mode=options.test_mode)
  File "./scripts/setup_irods.py", line 68, in setup_server
    check_hostname()
  File "./scripts/setup_irods.py", line 168, in check_hostname
    if not irods.lib.hostname_resolves_to_local_address(hostname):
  File "/home/syslab/workspace_chc/install_irods/var/lib/irods/scripts/irods/lib.py", line 386, in hostname_resolves_to_local_address
    hostname])
  File "/home/syslab/workspace_chc/install_irods/var/lib/irods/scripts/irods/execute.py", line 89, in execute_command_permissive
    p = execute_command_nonblocking(args, stdout=stdout, stderr=stderr, **kwargs)
  File "/home/syslab/workspace_chc/install_irods/var/lib/irods/scripts/irods/execute.py", line 64, in execute_command_nonblocking
    sys.exc_info()[2])
  File "/home/syslab/workspace_chc/install_irods/var/lib/irods/scripts/irods/execute.py", line 54, in execute_command_nonblocking
    return subprocess.Popen(args, **kwargs)
  File "/usr/lib/python2.7/subprocess.py", line 394, in __init__
    errread, errwrite)
  File "/usr/lib/python2.7/subprocess.py", line 1047, in _execute_child
    raise child_exception
IrodsError: Call to open process with ['/home/syslab/workspace_chc/install_irods/usr/sbin/hostname_resolves_to_local_address', 'syslab'] failed:
  Could not find the requested executable '/home/syslab/workspace_chc/install_irods/usr/sbin/hostname_resolves_to_local_address'; please ensure '/home/syslab/workspace_chc/install_irods/usr/sbin/hostname_resolves_to_local_address' is installed and in the path.

Thank you for reading it. I look forward to hearing from you.

Create icommand irmdir

Functionality would be the same as Unix rmdir.

Requested in irods/irods#3117.

missing irmdir in ihelp listing

icd not handling special characters correctly

We are using iRODS version 4.2.8. We have a collection whose name has special characters in it including new lines. When I attempt to use icd to navigate into it from its parent collection, icd changes the current working collection to my home collection and exits without error.

[irods@junior ~]$ ipwd
/iplant/home/username
[irods@junior ~]$ ils
/iplant/home/username:
  C- /iplant/home/username/ATGCC
+
::;4ADBD?FFB=?;C;:::<<FBE<8?8:11?B0?BD030?**
  C- /iplant/home/username/logs
[irods@junior ~]$ icd 'ATGCC
+
::;4ADBD?FFB=?;C;:::<<FBE<8?8:11?B0?BD030?**'
[irods@junior ~]$ echo $?
0
[irods@junior ~]$ ipwd
/iplant/home/rods

resource and user AVUs can only be updated by rodsadmins

document this in the imeta help text

Meaning of exit code output from ifsck

Hi Foks,

I noticed that when this command runs and completes successfully, it gives an exit code that is greater than 1 and I was wondering what the significance was.

$ ifsck -K -r /irods-seq-i23-bc/
$ echo $?
192

Extend ils -l to show system metadata about collections

One of our users has requested that the ils -l be extended show system metadata about collections. Maybe COLL_OWNER_NAME, COLL_OWNER_ZONE, and COLL_MODIFY_TIME values could be added to the output and displayed in a way that is similar to the corresponding data object values are displayed.

Bash autocomplete for iCommands not working right

I have iRODS 4.3.0 installed and found the bash completion script here. After sourcing it, it kind of works, but it seems to always think I'm in my home directory. For example:

$ ipwd <enter>
/iplant/home/chandles
$ ils <enter>
/iplant/home/chandles:
  C- /iplant/home/chandles/analyses
$ ils <tab>(autocompletes to:)
$ ils analyses/<tab><tab>(lists possible options:)
analyses/Cloud_Shell_analysis1-2022-03-11-17-53-14.3/  analyses/Datahog_analysis1-2022-09-06-18-51-07.8/      analyses/Datahog_analysis1-2022-09-14-21-51-41.6/

$ ils analyses/ <enter>
/iplant/home/chandles/analyses:
  C- /iplant/home/chandles/analyses/Cloud_Shell_analysis1-2022-03-11-17-53-14.3
  C- /iplant/home/chandles/analyses/Datahog_analysis1-2022-09-06-18-51-07.8
  C- /iplant/home/chandles/analyses/Datahog_analysis1-2022-09-14-21-51-41.6
$

So it worked ok there, but if i icd into analyses directory it still think I'm in my home dir:

$ icd <enter>
$ icd <tab>(autocompletes to:)
$ icd ./analyses/ <enter>
$ icd <tab>(autocompletes to:)
$ icd analyses/ <enter>
No such directory (collection): /iplant/home/chandles/analyses/analyses
$

If I try to see above, it doesn't do anything:

$ ils ../<tab><tab>(nothing happens)

Actually it gets hung here and I have to ^C. Further, if I icd into our shared community data, it still thinks i'm in my home directory:

$ ils ../^C^C
$ ^C
$ ^C
$ ipwd
/iplant/home/chandles/analyses
$ icd ../../shared/agi_data <enter>
$ icd <tab>(autocompletes to:)
$ icd analyses/ <enter>
No such directory (collection): /iplant/home/shared/agi_data/analyses
$ ils <enter>
/iplant/home/shared/agi_data:
  C- /iplant/home/shared/agi_data/Asian_rice_pangenome_annotations
  C- /iplant/home/shared/agi_data/Dr_Lorieux_Data
  C- /iplant/home/shared/agi_data/HiFI_DATA
  C- /iplant/home/shared/agi_data/KAUST
$ ils H<tab>(nothing happens)

Need to package icommands for NIX

I'm currently trying to create a NIX package of the irods-icommands to provide an easy way to deploy them to hundreds of heterogeneous computing nodes.
As I failed with irods 4.1.8, I was wondering if the new build system of irods 4.2, which seems to be based on cmake, can help me.
I'm facing a first issue:

bzizou@killeen:~/git/irods_client_icommands/build$` cmake ..                 
CMake Error at CMakeLists.txt:4 (find_package):                             
  By not providing "FindIRODS.cmake" in CMAKE_MODULE_PATH this project has  
  asked CMake to find a package configuration file provided by "IRODS", but 
  CMake did not find one.                                                   

  Could not find a package configuration file provided by "IRODS" (requested
  version 4.2.0) with any of the following names:                           

    IRODSConfig.cmake                                                       
    irods-config.cmake                                                      

  Add the installation prefix of "IRODS" to CMAKE_PREFIX_PATH or set        
  "IRODS_DIR" to a directory containing one of the above files.  If "IRODS" 
  provides a separate development package or SDK, be sure it has been       
  installed.                                                                


-- Configuring incomplete, errors occurred!

Is there any developer's documentation? I would be very glad to contribute, I think that having Nix packages for the irods clients could be a great thing to bring irods to any kind of unix client hosts.

irepl does not processing remaining existing data objects if it processes a nonexistent one

I'm using iCommands version 4.2.8.

When irepl is provided multiple data objects on its command line, it processes them in the order they appear on the command line. If it processes a data object that does not exist, it keeps processing the remaining until it finds one that exists. Before it processes the one that exists, it exits. Here's an example.

ipc_admin@prod ~? irepl -B -M -v -RtaccRes /iplant/home/user/missing1 /iplant/home/user/exists /iplant/home/user/missing2 
remote addresses: 206.207.252.46 ERROR: replUtil: srcPath /iplant/home/home/user/missing1 does not exist
remote addresses: 206.207.252.46 ERROR: replUtil: srcPath /iplant/home/home/user/missing2 does not exist
remote addresses: 206.207.252.46 ERROR: replUtil: invalid repl objType 0 for /iplant/home/user/missing1

ipc_admin@prod ~? ils /iplant/home/user/missing1 /iplant/home/user/missing2
remote addresses: 206.207.252.46 ERROR: lsUtil: srcPath /iplant/home/user/missing1 does not exist or user lacks access permission
remote addresses: 206.207.252.46 ERROR: lsUtil: srcPath /iplant/home/user/missing2 does not exist or user lacks access permission

ipc_admin@prod ~? ils -l /iplant/home/user/exists
  user          0 CyVerseRes;ds16Broker;ds16          432 2021-06-29.15:13 & exists

Other problems, like a data object missing its replica, don't cause this exit early behavior.

This behavior makes it hard to script bulk irepl operations. When a failure occurs, it's difficult to figure out which data objects weren't processed. It would be nice if irepl always attempted every data object on its command line, reporting data objects that it couldn't find like it reports other problems.

Which commit on 4-2-stable should be used to build clients for iRODS 4.2.7?

Which commit on the 4-2-stable branch is the correct one to build iRODS client compatible with iRODS 4.2.7? The HEAD doesn't appear to be compatible any more. There are a number of commits that work, but it's not clear which is preferred, or correct.

Please consider tagging 4-2-stable with the corresponding release number. It would be very helpful when reproducing a build.

Thanks.

imeta ignored return values

E.g. https://github.com/irods/irods_client_icommands/blob/4.2.1/src/imeta.cpp#L1446

This error is repeated throughout the file.

imeta mod does not work in 4.2.1 and 4.2.2

imeta mod throws error when modifying metadata (icommands v 4.2.1 or 4.2.2):

imeta mod -d test.txt KEY value v:myValue
remote addresses: irods-server ERROR: rcModAVUMetadata failed with error -816000 CAT_INVALID_ARGUMENT

while with icommands v 4.1.10 and higher the command works as expected.

irsync -r --link fails on a bad soft link

This is related to issue #21.