The XTestCase.getTestUser() method returns the hardcoded username test. This results in failure due to SSH authentication failure in case this user is not setup correctly. about oozie

[aprabhakar]

Here is a patch that implements the second approach:

$ git diff --no-prefix --cached
diff --git core/src/test/java/org/apache/oozie/test/XTestCase.java core/src/test/java/org/apache/oozie/test/XTestCase.java 
index 014e581..4f3bb69 100644
--- core/src/test/java/org/apache/oozie/test/XTestCase.java
+++ core/src/test/java/org/apache/oozie/test/XTestCase.java
@@ -153,7 +153,7 @@ public abstract class XTestCase extends TestCase {
     }

     protected String getTestUser() {
-        return "test";
+        return System.getProperty("user.name", "test");
     }

     protected String getTestUser2() {
    diff --git pom.xml pom.xml
    index 75d70c8..f1ab828 100644
--- pom.xml
+++ pom.xml
@@ -219,6 +219,10 @@
                     <argLine>-Xmx512m</argLine>
                     <systemProperties>
                         <property>
+                            <name>user.name</name>
+                            <value>${user.name}</value>
+                        </property>
+                        <property>
                             <name>oozie.test.dir</name>
                             <value>${oozie.test.dir}</value>
                         </property>

[tucu00]

using ${user.name} will break things.
specially with Hadoop 20S.
${user.name} user is used to run Oozie.
'test', 'test2' and 'test3' users are used as regular users for testcases.
in addition there should be a group 'testg' to which 'test' and 'test2' users must belong and 'test3' must not belong.
these users&group should be defined in the system (Hadoop looks for users/groups at OS level)

[aprabhakar]

The username can be overwritten by specifying -Duser.name=test. I tested that out with Java 6 and it seems to work. If that does not work, can you suggest an alternative?

[tucu00]

for H20.2 things will work, but won't with H20S

[aprabhakar]

Instead, can we do this: allow the passing of the user name only if a specific property is set (say test.user.name). That way, the Yahoo! tests can continue as they are and individual developers can override the test name where needed. Here is a patch that implements it:

diff --git core/src/test/java/org/apache/oozie/test/XTestCase.java core/src/test/java/org/apache/oozie/test/XTestCase.java
index 014e581..b3299ef 100644
--- core/src/test/java/org/apache/oozie/test/XTestCase.java
+++ core/src/test/java/org/apache/oozie/test/XTestCase.java
@@ -93,6 +93,12 @@ public abstract class XTestCase extends TestCase {
      */
     public static final String HADOOP_VERSION = "hadoop.version";

+    /** 
+     * System property to specify the main test user name.
+     * If this property is not set, the default test user name is used which * is <tt>test</tt>.
+     */
+    public static final String TEST_USER_NAME_PROP = "test.user.name";
+
     /**
      * Initialize the test working directory. <p/> If it does not exist it creates it, if it already exists it deletes
      * all its contents. <p/> The test working directory it is not deleted after the test runs. <p/>
@@ -153,6 +159,10 @@ public abstract class XTestCase extends TestCase {
     }

     protected String getTestUser() {
+        String testUserName = System.getProperty(TEST_USER_NAME_PROP);
+        if (testUserName != null && !testUserName.equals("${"+TEST_USER_NAME_PROP+"}")) {
+          return testUserName;
+        }
         return "test";
     }

diff --git pom.xml pom.xml
index 75d70c8..a1c68e5 100644
--- pom.xml
+++ pom.xml
@@ -219,6 +219,10 @@
                     <argLine>-Xmx512m</argLine>
                     <systemProperties>
                         <property>
+                            <name>test.user.name</name>
+                            <value>${test.user.name}</value>
+                        </property>
+                        <property>
                             <name>oozie.test.dir</name>
                             <value>${oozie.test.dir}</value>
                         </property>

[tucu00]

Yes, we can do something like that.

But I'd prefer to do this for ALL test users and groups. We've got bitten more than once with the subtle changes in how Hadoop takes users and groups through out versions (18, 19, 20, 20S).

I'd propose the following parameterization:

oozie.test.user.oozie=${user.name}
oozie.test.user.test=test
oozie.test.user.test1=test1
oozie.test.user.test2=test2
oozie.test.group=testg

[aprabhakar]

Sounds good. Here is a patch that does that:

diff --git core/src/test/java/org/apache/oozie/test/XTestCase.java core/src/test/java/org/apache/oozie/test/XTestCase.java
index 014e581..90529bd 100644
--- core/src/test/java/org/apache/oozie/test/XTestCase.java
+++ core/src/test/java/org/apache/oozie/test/XTestCase.java
@@ -93,6 +93,36 @@ public abstract class XTestCase extends TestCase {
      */
     public static final String HADOOP_VERSION = "hadoop.version";

+    /** 
+     * System property that specifies the user that test oozie instance runs as.
+     * The value of this property defaults to the "${user.name} system property.
+     */
+    public static final String TEST_OOZIE_USER_PROP = "oozie.test.user.oozie";
+
+    /**
+     * System property that specifies the default test user name used by 
+     * the tests. The defalt value of this property is <tt>test</tt>.
+     */
+    public static final String TEST_USER1_PROP = "oozie.test.user.test";
+
+    /**
+     * System property that specifies an auxilliary test user name used by the 
+     * tests. The default value of this property is <tt>test2</tt>.
+     */
+    public static final String TEST_USER2_PROP = "oozie.test.user.test2";
+
+    /**
+     * System property that specifies another auxilliary test user name used by
+     * the tests. The default value of this property is <tt>test3</tt>.
+     */
+    public static final String TEST_USER3_PROP = "oozie.test.user.test3";
+
+    /**
+     * System property that specifies the test groiup used by the tests.
+     * The default value of this property is <tt>testg</tt>.
+     */
+    public static final String TEST_GROUP_PROP = "oozie.test.group";
+
     /**
      * Initialize the test working directory. <p/> If it does not exist it creates it, if it already exists it deletes
      * all its contents. <p/> The test working directory it is not deleted after the test runs. <p/>
@@ -152,25 +182,39 @@ public abstract class XTestCase extends TestCase {
         return hadoopVersion;
     }

+    public static String getOozieUser() {
+        return getSystemProperty(TEST_OOZIE_USER_PROP, 
+            System.getProperty("user.name"));
+    }
+
+    /**
+     * @return the user to use in testcases.
+     */
     protected String getTestUser() {
-        return "test";
+        return getSystemProperty(TEST_USER1_PROP, "test");
     }

+    /**
+     * @return the user to use in testcases.
+     */
     protected String getTestUser2() {
-        return "test2";
+        return getSystemProperty(TEST_USER2_PROP, "test2");
     }

+    /**
+     * @return the user to use in testcases.
+     */
     protected String getTestUser3() {
-        return "test3";
+        return getSystemProperty(TEST_USER3_PROP, "test3");
     }

     /**
-     * Return the user to use in testcases.
+     * Return the user group to use in testcases.
      *
-     * @return the user to use in testcases.
+     * @return the user group to use in testcases.
      */
     protected String getTestGroup() {
-        return "testg";
+        return getSystemProperty(TEST_GROUP_PROP, "testg");
     }

     /**
@@ -345,6 +389,21 @@ public abstract class XTestCase extends TestCase {
         return System.getProperty(OOZIE_TEST_NAME_NODE, "hdfs://localhost:9000");
     }

+    /**
+     * Returns the value of the system property specifiedy by the 
+     * <tt>propertyName</tt>. If the property value is not set or its value 
+     * is equal to the string <tt>${propertyName}</tt>, the default value
+     * is returned.
+     */
+    public static String getSystemProperty(String propertyName, String defaultValue) {
+        String value =  System.getProperty(propertyName, defaultValue);
+        if ( value != null && value.equals("${" + propertyName + "}")) {
+            // Value was not substituted, use the default instead
+            return defaultValue;
+        }
+        return value;
+    }
+
     public String getKeytabFile() {
         String defaultFile = new File(System.getProperty("user.home"), "oozie.keytab").getAbsolutePath();
         return System.getProperty("oozie.test.kerberos.keytab.file", defaultFile);
@@ -356,7 +415,7 @@ public abstract class XTestCase extends TestCase {

     public String getOoziePrincipal() {
         return System.getProperty("oozie.test.kerberos.oozie.principal",
-                                  System.getProperty("user.name") + "/localhost") + "@" + getRealm();
+                                  getOozieUser() + "/localhost") + "@" + getRealm();
     }

     public String getJobTrackerPrincipal() {
@@ -500,12 +559,13 @@ public abstract class XTestCase extends TestCase {
             }
             int taskTrackers = 2;
             int dataNodes = 2;
+            String oozieUser = getOozieUser();
             JobConf conf = new JobConf();
             conf.set("dfs.block.access.token.enable", "false");
             conf.set("dfs.permissions", "true");
             conf.set("hadoop.security.authentication", "simple");
-            conf.set("hadoop.proxyuser." + System.getProperty("user.name") + ".hosts", "localhost");
-            conf.set("hadoop.proxyuser." + System.getProperty("user.name") + ".groups", "users");
+            conf.set("hadoop.proxyuser." + oozieUser + ".hosts", "localhost");
+            conf.set("hadoop.proxyuser." + oozieUser + ".groups", "users");
             conf.set("mapred.tasktracker.map.tasks.maximum", "4");
             conf.set("mapred.tasktracker.reduce.tasks.maximum", "4");
             dfsCluster = new MiniDFSCluster(conf, dataNodes, true, null);
diff --git pom.xml pom.xml
index 75d70c8..76b9a9f 100644
--- pom.xml
+++ pom.xml
@@ -219,6 +219,26 @@
                     <argLine>-Xmx512m</argLine>
                     <systemProperties>
                         <property>
+                            <name>oozie.test.user.oozie</name>
+                            <value>${oozie.test.user.oozie}</value>
+                        </property>
+                        <property>
+                            <name>oozie.test.user.test</name>
+                            <value>${oozie.test.user.test}</value>
+                        </property>
+                        <property>
+                            <name>oozie.test.user.test2</name>
+                            <value>${oozie.test.user.test2}</value>
+                        </property>
+                        <property>
+                            <name>oozie.test.user.test3</name>
+                            <value>${oozie.test.user.test3}</value>
+                        </property>
+                        <property>
+                            <name>oozie.test.group</name>
+                            <value>${oozie.test.group}</value>
+                        </property>
+                        <property>
                             <name>oozie.test.dir</name>
                             <value>${oozie.test.dir}</value>
                         </property>

I have changed the test1 property and value to test2 an d test2 property and value to test3 for consistency with the current implementation. They can be changed back if necessary.

[tucu00]

Looks good.

One question though, the getSystemProperty() method, in which use case it would happen that the the value is '${propertyName}' ?

We'll add this to the next drop

[aprabhakar]

Regarding the ${propertyName} as literal value - this happens when the property is set to be passed via the surefire plugin but has not been specified using the -D option. In that case, it seems to pass the whole literal as its value and can cause confusion. Hence the check.

[tucu00]

To avoid that the pom.xml must have in the top properties the following and you should be done.

.. ${user.name} test test1 test2 test2

[aprabhakar]

That sounds good too - I will update the patch/diff and attach it tomorrow.

[aprabhakar]

Here is the updated patch:

diff --git core/src/test/java/org/apache/oozie/test/XTestCase.java core/src/test/java/org/apache/oozie/test/XTestCase.java
index 014e581..b9d8f8e 100644
--- core/src/test/java/org/apache/oozie/test/XTestCase.java
+++ core/src/test/java/org/apache/oozie/test/XTestCase.java
@@ -93,6 +93,36 @@ public abstract class XTestCase extends TestCase {
      */
     public static final String HADOOP_VERSION = "hadoop.version";

+    /** 
+     * System property that specifies the user that test oozie instance runs as.
+     * The value of this property defaults to the "${user.name} system property.
+     */
+    public static final String TEST_OOZIE_USER_PROP = "oozie.test.user.oozie";
+
+    /**
+     * System property that specifies the default test user name used by 
+     * the tests. The defalt value of this property is <tt>test</tt>.
+     */
+    public static final String TEST_USER1_PROP = "oozie.test.user.test";
+
+    /**
+     * System property that specifies an auxilliary test user name used by the 
+     * tests. The default value of this property is <tt>test2</tt>.
+     */
+    public static final String TEST_USER2_PROP = "oozie.test.user.test2";
+
+    /**
+     * System property that specifies another auxilliary test user name used by
+     * the tests. The default value of this property is <tt>test3</tt>.
+     */
+    public static final String TEST_USER3_PROP = "oozie.test.user.test3";
+
+    /**
+     * System property that specifies the test groiup used by the tests.
+     * The default value of this property is <tt>testg</tt>.
+     */
+    public static final String TEST_GROUP_PROP = "oozie.test.group";
+
     /**
      * Initialize the test working directory. <p/> If it does not exist it creates it, if it already exists it deletes
      * all its contents. <p/> The test working directory it is not deleted after the test runs. <p/>
@@ -152,25 +182,39 @@ public abstract class XTestCase extends TestCase {
         return hadoopVersion;
     }

+    public static String getOozieUser() {
+        return System.getProperty(TEST_OOZIE_USER_PROP, 
+            System.getProperty("user.name"));
+    }
+
+    /**
+     * @return the user to use in testcases.
+     */
     protected String getTestUser() {
-        return "test";
+        return System.getProperty(TEST_USER1_PROP, "test");
     }

+    /**
+     * @return the user to use in testcases.
+     */
     protected String getTestUser2() {
-        return "test2";
+        return System.getProperty(TEST_USER2_PROP, "test2");
     }

+    /**
+     * @return the user to use in testcases.
+     */
     protected String getTestUser3() {
-        return "test3";
+        return System.getProperty(TEST_USER3_PROP, "test3");
     }

     /**
-     * Return the user to use in testcases.
+     * Return the user group to use in testcases.
      *
-     * @return the user to use in testcases.
+     * @return the user group to use in testcases.
      */
     protected String getTestGroup() {
-        return "testg";
+        return System.getProperty(TEST_GROUP_PROP, "testg");
     }

     /**
@@ -356,7 +400,7 @@ public abstract class XTestCase extends TestCase {

     public String getOoziePrincipal() {
         return System.getProperty("oozie.test.kerberos.oozie.principal",
-                                  System.getProperty("user.name") + "/localhost") + "@" + getRealm();
+                                  getOozieUser() + "/localhost") + "@" + getRealm();
     }

     public String getJobTrackerPrincipal() {
@@ -500,12 +544,13 @@ public abstract class XTestCase extends TestCase {
             }
             int taskTrackers = 2;
             int dataNodes = 2;
+            String oozieUser = getOozieUser();
             JobConf conf = new JobConf();
             conf.set("dfs.block.access.token.enable", "false");
             conf.set("dfs.permissions", "true");
             conf.set("hadoop.security.authentication", "simple");
-            conf.set("hadoop.proxyuser." + System.getProperty("user.name") + ".hosts", "localhost");
-            conf.set("hadoop.proxyuser." + System.getProperty("user.name") + ".groups", "users");
+            conf.set("hadoop.proxyuser." + oozieUser + ".hosts", "localhost");
+            conf.set("hadoop.proxyuser." + oozieUser + ".groups", "users");
             conf.set("mapred.tasktracker.map.tasks.maximum", "4");
             conf.set("mapred.tasktracker.reduce.tasks.maximum", "4");
             dfsCluster = new MiniDFSCluster(conf, dataNodes, true, null);
diff --git pom.xml pom.xml
index 75d70c8..99fcc8c 100644
--- pom.xml
+++ pom.xml
@@ -58,6 +58,11 @@
         <oozie.test.kerberos.oozie.principal>${user.name}/localhost</oozie.test.kerberos.oozie.principal>
         <oozie.test.kerberos.jobtracker.principal>mapred/localhost</oozie.test.kerberos.jobtracker.principal>
         <oozie.test.kerberos.namenode.principal>hdfs/localhost</oozie.test.kerberos.namenode.principal>
+        <oozie.test.user.oozie>${user.name}</oozie.test.user.oozie>
+        <oozie.test.user.test>test</oozie.test.user.test>
+        <oozie.test.user.test2>test2</oozie.test.user.test2>
+        <oozie.test.user.test3>test3</oozie.test.user.test3>
+        <oozie.test.group>testg</oozie.test.group>
     </properties>

     <licenses>
@@ -219,6 +224,26 @@
                     <argLine>-Xmx512m</argLine>
                     <systemProperties>
                         <property>
+                            <name>oozie.test.user.oozie</name>
+                            <value>${oozie.test.user.oozie}</value>
+                        </property>
+                        <property>
+                            <name>oozie.test.user.test</name>
+                            <value>${oozie.test.user.test}</value>
+                        </property>
+                        <property>
+                            <name>oozie.test.user.test2</name>
+                            <value>${oozie.test.user.test2}</value>
+                        </property>
+                        <property>
+                            <name>oozie.test.user.test3</name>
+                            <value>${oozie.test.user.test3}</value>
+                        </property>
+                        <property>
+                            <name>oozie.test.group</name>
+                            <value>${oozie.test.group}</value>
+                        </property>
+                        <property>
                             <name>oozie.test.dir</name>
                             <value>${oozie.test.dir}</value>
                         </property>

I removed the getSystemProperty() method altogether as it was not serving any purpose anymore.

[rvs]

Arvind, could you, please fork and make this patch available in a branch called gh-3 ?

It is somewhat inconvenient to review patches embedded inside of GH issues.

Thanks,
Roman.

[tucu00]

R, patch is avail at tucu00/oozie GH-3