Make config/aws.yml task-specific about skypilot

[infwinston]

We need a better design to fix this issue. Things would fail if users simultaneously running two python examples/resnet_app.py.

[infwinston]

I'm currently working on sky-executing multiple independent tasks, where each task relies on a different yml config file.

[concretevitamin]

@gmittal and I chatted about this; Gautam, are you taking charge of designing this within the CLI?

We need to support both requirements:

• Support launching scripts on different clusters
• Support launching scripts on the same cluster. Mostly for debugging.

We can take inspiration from the Ray CLI:

ray up cluster.yaml
ray up cluster.yaml  # Same cluster is used.
ray up cluster.yaml -n <name>  # Override the cluster name -> a new cluster is used.

So, we can for example do

sky run <script or app>.py 
sky run <script or app>.py  # Same cluster is used.
sky run <script or app>.py --cluster=<name>  # A new cluster is used.

# However, it remains to be seen how to conveniently override these,
# where sky.execute(dag) is called from within:
python app.py 

[gmittal]

In #58 there's a fix for this. Any feedback would be appreciated.

[concretevitamin]

Fixed by CLI.