azure / machinelearningnotebooks

Python notebooks with ML and deep learning examples with Azure Machine Learning Python SDK | Microsoft

Azure Machine Learning Python SDK notebooks

With the introduction of AzureML SDK v2, this samples repository for the v1 SDK is now deprecated and will not be monitored or updated. Users are encouraged to visit the v2 SDK samples repository instead for up-to-date and enhanced examples of how to build, train, and deploy machine learning models with AzureML's newest features.

Welcome to the Azure Machine Learning Python SDK notebooks repository!

Getting started

These notebooks are recommended for use in an Azure Machine Learning Compute Instance, where you can run them without any additional set up.

However, the notebooks can be run in any development environment with the correct azureml packages installed.

Install the azureml.core Python package:

pip install azureml-core

Install additional packages as needed:

pip install azureml-mlflow
pip install azureml-dataset-runtime
pip install azureml-automl-runtime
pip install azureml-pipeline
pip install azureml-pipeline-steps
...

We recommend starting with one of the quickstarts.

Contributing

This repository is a push-only mirror. Pull requests are ignored.

Code of Conduct

This project has adopted the Microsoft Open Source Code of Conduct. Please see the code of conduct for details.

Reference

• Documentation

machinelearningnotebooks's Issues

how to install Databricks_AMLSDK_github.dbc on DB?

I don't know how to install Databricks_AMLSDK_github.dbc on DB. I've tried 'import library' on DB but dbc file is not supported.

02.deploy-models: AzureMLException: Illegal state. Unpack=False

Running cell 3 of 02.deploy-models (under "Retrieve the model"), gives this error if there is an existing sklearn_mnist_model.pkl file in the folder:

---------------------------------------------------------------------------
AzureMLException                          Traceback (most recent call last)
<ipython-input-13-8f39eeb11923> in <module>
      8 import os
      9 #os.remove('./sklearn_mnist_model.pkl')
---> 10 model.download(target_dir='.', exists_ok=True)
     11 
     12 # verify the downloaded model file

~\AppData\Local\conda\conda\envs\aml\lib\site-packages\azureml\core\model.py in download(self, target_dir, exists_ok)
    506         if len(file_paths) != 1:
    507             raise AzureMLException(
--> 508                 "Illegal state. Unpack={}, Paths in target_dir is {}".format(self.unpack, file_paths))
    509         return os.path.join(target_dir, file_paths[0])
    510 

AzureMLException: Illegal state. Unpack=False, Paths in target_dir is []

This is due to a bug in Model.download. As a workaround, you can delete the file first:

from azureml.core import Workspace
from azureml.core.model import Model

ws = Workspace.from_config()
model=Model(ws, 'sklearn_mnist') # get the latest version

import os 
os.remove('./sklearn_mnist_model.pkl')
model.download(target_dir='.', exists_ok=True)

# verify the downloaded model file
os.stat('./sklearn_mnist_model.pkl')

Errors in your scoring file's init() function

Hi, trying to deploy a simple text classifier model using sklearn tfidf and LinearSVC. Managed to register models, create environment file, create image but deployment fails, suggesting there might be an issue with the init() function.

{'code': 'AciDeploymentFailed', 'message': 'Aci Deployment failed', 'details': [{'code': 'CrashLoopBackOff', 'message': "Your container application crashed. This may be caused by errors in your scoring file's init() function.\nPlease check the logs for your container instance high-priority.\nYou can also try to run image highpriority7173083745.azurecr.io/high-priority:7 locally. Please refer to http://aka.ms/debugimage for more information."}]}

Tested score.py locally loading persisted model from local via joblib and returns predictions as expected so nothing seems wrong with the script or models.

Any ideas?!

Here is the score.py code:

import pickle
import json
import numpy as np
import os
from sklearn.externals import joblib
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.svm import LinearSVC

from azureml.core.model import Model

def init():
    global model_1,model_2
    # get model path
    model_path1 = Model.get_model_path("tfidf_hp")
    model_path2 = Model.get_model_path("LinearSVC_hp")
    # deserialize the model file back into a sklearn model
    model_1,model_2 = joblib.load(model_path1), joblib.load(model_path2)


def run(raw_data):
    # make prediction
    data = json.loads(raw_data)['data']
    y_pred = model_2.predict(model_1.transform(data))
    return json.dumps({"prediction": y_pred.tolist()})

Import problem in 03.auto-train-models.ipynb notebook

When running 03.auto-train-models.ipynb in azure notebooks, the import azureml.core fails even-though ! pip freeze | grep azureml returns azureml==0.2.7, the kernel is Python 3 it seems that 3.6 is not available. Thanks

Question: GPU enabled webservice

Hi,

I was wondering what the best option for inference with a model is. I have trained a Faster R-CNN model locally with CNTK and on my laptop with a GT940m and a i7 4700HQ there is a significant difference in speed when running on CPU vs GPU (ofcourse). Now I only saw utilizing a CPU instance for a webservice in the examples. I would like to use a GPU to run inference on images, behind an endpoint and not too expensive it that's possible. Any tips?

root_mean_squared_error causing 'parentRun configuration is invalid.' ?

while trying to use root_mean_squared_error on regression task got this error
(doesnt happen with different metrics?)

<class 'azureml.train.automl.exceptions.ServiceException'>: Unknown error raised. An unknown error has occurred.
<traceback object at 0x7fb9b64a4388>

~/anaconda3_501/lib/python3.6/site-packages/azureml/_restclient/operations/jasmine_operations.py in create_parent_run(self, subscription_id, resource_group_name, workspace_name, project_name, experiment_name, create_parent_run_dto, custom_headers, raw, **operation_config)
368 if response.status_code not in [200]:
--> 369 raise HttpOperationError(self._deserialize, response)
370

HttpOperationError: Operation returned an invalid status code 'parentRun configuration is invalid.'

Error In 01.train-within-notebook, Model.list fails

On the step when you verify that model has been registered properly , Model.list seems to be giving below error. Wondering if the parameters are updated ?

from azureml.core.model import Model
models = Model.list(name='best_model')
for m in models:
print(m.name, m.version)

---

TypeError Traceback (most recent call last)
in ()
1 from azureml.core.model import Model
----> 2 models = Model.list(name='best_model')
3 for m in models:
4 print(m.name, m.version)

TypeError: list() missing 1 required positional argument: 'workspace'

ImportError: No module named 'azureml.core'

I am doing AML tutorial in 03.auto-train-models.ipynb.
As a very new user, I get stuck from the beginning.

From Prerequisites
I am creating a workspace and its configuration file (config.json)
I create python code and run in Azure ML workbench as below: (xxx is hidden info)

#######
from azureml.core import Workspace
subscription_id ='xxxxxxx-xxxx-xxxx-bf6a-e03e4fe8bced'
resource_group ='xxxxxxx-AZURE-DEV'
workspace_name = 'xxxxxx-test-AML'

try:
ws = Workspace(subscription_id = subscription_id, resource_group = resource_group, workspace_name = workspace_name)
ws.write_config()
print('Library configuration succeeded')
except:
print('Workspace not found')

#######

After I run the script, there is an error below:

File "config.py", line 1, in
from azureml.core import Workspace
ImportError: No module named 'azureml.core'

Could you help me what did I do wrong?
It seem that azureml library is not connected.
by the way, I have installed azureml and confirm with pip freeze. (azureml==0.2.7)

conda version : 4.5.11
python version : 3.6.5.final.0
platform : win-64

Possible double serialization on results returned from web service

Noticed strange behavior on results returned from web service after constructing score.py following the SDK documentation. See the below comparison.

Attempt 1. run() function using json.dumps on return:

def run(input_json):
    '''
    Input: JSON "{\"skill\": \"legal\"}"
    '''

    try:
        input_df = json.loads(input_json)["skill"].lower()
        assert len(input_df)>0
    except:
        print("Input data is not in JSON format")

    res = predict(input_df)
    res_dict = {"skills" : res}

    return json.dumps(res_dict)

Output (notice the excessive backslashes):

"{\"skills\": [\"finance\", \"corporate_finance\"]}"

Attempt 2. run() function simply returning Python dictionary:

def run(input_json):
    '''
    SAME CODE
    '''
    return res_dict

Output:

{"skills": ["finance", "corporate_finance"]}

The first output is consistent with applying json.dumps twice.

print(json.dumps(json.dumps({"skills":["finance","corporate_finance"]})))

Result:

"{\"skills\": [\"finance\", \"corporate_finance\"]}"

Makes me wonder if the API service is doing its own json.dumps on returned object already? Hence there should be no need of adding json.dumps in the run() function.

automl: are categorical columns supported?

Hi, just testing a dataframe with mostly cat cols on pandas, is this supported already? Thanks!

With preprocess=True

            Automl_config = AutoMLConfig(
task='regression',
preprocess = True,
max_time_sec=600,
iterations=100,
primary_metric='normalized_root_mean_squared_log_error',
X=X, 
y=y,
n_cross_validations=5)

~/anaconda3_501/lib/python3.6/site-packages/pandas/core/generic.py in getattr(self, name)
3079 if name in self._info_axis:
3080 return self[name]
-> 3081 return object.getattribute(self, name)
3082
3083 def setattr(self, name, value):

AttributeError: 'Series' object has no attribute 'isna'

Without preprocess

            Automl_config = AutoMLConfig(
task='regression',
preprocess = False,
max_time_sec=600,
iterations=100,
primary_metric='normalized_root_mean_squared_log_error',
X=X, 
y=y,
n_cross_validations=5)

Parent Run ID: AutoML_d448e304-8e28-4a3d-a9da-76af13b0f46c

---

ITERATION: The iteration being evaluated.
PIPELINE: A summary description of the pipeline being evaluated.
DURATION: Time taken for the current iteration.
METRIC: The result of computing score on the fitted pipeline.
BEST: The best observed score thus far.

---

ITERATION PIPELINE DURATION METRIC BEST
0 0:00:09.530248 179769313486231570814527423731704356798070567525844996598917476803157260780028538760589558632766878171540458953514382464234321326889464182768467546703537516986049910576551282076245490090389328944075868508455133942304583236903222948165808559332123348274797826204144723168738177180919299881250404026184124858368.000179769313486231570814527423731704356798070567525844996598917476803157260780028538760589558632766878171540458953514382464234321326889464182768467546703537516986049910576551282076245490090389328944075868508455133942304583236903222948165808559332123348274797826204144723168738177180919299881250404026184124858368.000
ERROR: ufunc 'isnan' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe''

Adding custom files (own modules) in docker image

Hi,

I want to deploy my own trained model like stated in this notebook (https://github.com/Azure/MachineLearningNotebooks/blob/master/tutorials/02.deploy-models.ipynb)

When I want to make the docker file, it seems that it's only possible using the Environment .yml file. Though to score on my model I need more then some packages, but I need to use my own local and custom models. Basically, I want to upload some extra folders to my dockerImage so that I can import that code in score.py.

Is there any way to add files in the docker of do I have to put them online so I can download them in the init() phase? Which would be a very inefficient workaround...

Regards

Deploying Faster R-CNN tensorflow model

This is not an issue as such. I've been trying to deploy a frozen inference graph I generated using the Tensorflow model zoo faster rcnn inception v2 model. So far I've encounted issues with a dependency on Open CV which I fixed by shifting to use of Pillow but I'm still not able to get the initialization of my score.py working. I wondered if there was any advice on what I'm attempting, particularly if what I'm trying to do is, in fact, not yet supported. I'd be most willing to share my experience and any insights I've gained if that helps in any way.

Error"Framework PyTorch with communicator None is not supported on BatchAI."

Hi,

I'm trying to use HyperDrive run to do the hyperparameter sweep work for Faster-RCNN model. I'm using CNTK instead of PyTorch model.

My customized container works fine when I submit a single experiment on BatchAI compute target, but when I wrapped it into a HyperDrive run, I got following error message.

Not sure how PyTorch came into the picture, I don't use it in my script.

Thanks
Shu

"[2018-11-12T22:47:41.449665][API][INFO]Experiment created\n""[2018-11-12T22:47:41.671104][GENERATOR][INFO]Trying to sample '1' jobs from the hyperparameter space\n""[2018-11-12T22:47:41.757886][GENERATOR][INFO]Successfully sampled '1' jobs, they will soon be submitted to the execution target.\n"[2018-11-12T22:47:54.9298488Z][SCHEDULER][ERROR]Exception occurred while preparing the environment. Message='{"Error":{"Code":"UserError","Message":"Framework PyTorch with communicator None is not supported on BatchAI.","Target":null,"Details":[],"InnerError":null,"DebugInfo":{"Type":"Microsoft.MachineLearning.Common.WebApi.Exceptions.BadRequestException","Message":"Framework PyTorch with communicator None is not supported on BatchAI.","StackTrace":"...skipped...","InnerException":null,"Data":{},"ErrorResponse":null}},"Correlation":{"operation":"f87346ff-464908009096e2d8","request":"bXr9tPY0hJE="}}'. InnerException=''. Stack trace:' at Microsoft.MachineLearning.Execution.Services.BatchAiStrategy.CreateUnderlyingJob(StrategyState initialState, DynamicWorkingRunState workingState)...."

WebserviceException not imported in project-brainwave-custom-weights.ipynb

Second cell under heading "Deploy":

from azureml.core.model import Model
from azureml.core.image import Image
from azureml.core.webservice import Webservice
from azureml.contrib.brainwave import BrainwaveWebservice, BrainwaveImage

model_name = "catsanddogs-densenet121-model"
image_name = "catsanddogs-densenet121-image"
service_name = "modelbuild-service"

registered_model = Model.register(ws, model_def_path, model_name)

image_config = BrainwaveImage.image_configuration()
deployment_config = BrainwaveWebservice.deploy_configuration()
    
try:
    service = Webservice(ws, service_name)
    service.delete()
    service = Webservice.deploy_from_model(ws, service_name, [registered_model], image_config, deployment_config)
except WebserviceException:
    service = Webservice.deploy_from_model(ws, service_name, [registered_model], image_config, deployment_config)

WebserviceException is never imported, so the cell fails if this exception is thrown.

Missing module - No module named 'automl.client'

@rastala
There is an error in the following notebook 03b.auto-ml-remote-batchai

when running the cell under" Retrieve the Best Model"

best_run, fitted_model = remote_run.get_output()

print(best_run)

print(fitted_model)

---

ModuleNotFoundError Traceback (most recent call last)
in ()
----> 1 best_run, fitted_model = remote_run.get_output()
2 print(best_run)
3 print(fitted_model)

~/anaconda3_501/lib/python3.6/site-packages/azureml/train/automl/run.py in get_output(self, iteration, metric)
329 curr_run.download_file(name=model_name, output_file_path=model_local)
330 with open(model_local, "rb") as model_file:
--> 331 fitted_model = pickle.load(model_file)
332 return curr_run, fitted_model
333

ModuleNotFoundError: No module named 'automl.client'

Docker Image Layer optimization

Hello,

When I'm using the SDK to create a docker image. The model is copied before installing the conda environment. this means that when only the model is changing, the storage for the environment will be duplicated in the docker registry.

IMAGE               CREATED             CREATED BY                                      SIZE                COMMENT
70c7ed31b44c        5 minutes ago       /bin/sh -c #(nop)  CMD ["supervisord" "-c" "…   0B
<missing>           5 minutes ago       /bin/sh -c pip --no-cache-dir install -r /va…   1.28MB
<missing>           5 minutes ago       /bin/sh -c conda env update --name=root -f /…   759MB               Environment installation
<missing>           8 minutes ago       /bin/sh -c #(nop) COPY dir:1fce004162580bc9b…   1.01MB              Model copy
<missing>           4 weeks ago         |1 PYTHON_VERSION=3.5.* /bin/sh -c chmod 755…   59B

I'm wasting 759MB (213MB when compressed) for each model whereas it could be shared if the environment installation layer is done before copying the model.

Thanks a lot!

ADB example 5 is missing

There is no example notebook 5 at https://github.com/Azure/MachineLearningNotebooks/tree/master/how-to-use-azureml/azure-databricks

AttributeError: 'GenericResource' object has no attribute 'workspaceid' ( v.0.1.80 )

Traceback (most recent call last):
File "C:\Anaconda3\lib\site-packages\azureml\core\workspace.py", line 822, in datastores
return {datastore.name: datastore for datastore in _DatastoreClient.list(self)}
File "C:\Anaconda3\lib\site-packages\azureml\data\datastore_client.py", line 216, in list
dss, ct = _DatastoreClient._list(workspace, ct, 100)
File "C:\Anaconda3\lib\site-packages\azureml\data\datastore_client.py", line 342, in _list
client = _DatastoreClient._get_client(ws, auth, host)
File "C:\Anaconda3\lib\site-packages\azureml\data\datastore_client.py", line 373, in _get_client
ws._subscription_id, ws._resource_group, ws._workspace_name), ws._workspace_id)
File "C:\Anaconda3\lib\site-packages\azureml\core\workspace.py", line 928, in _workspace_id
return self._workspace_autorest_object.workspaceid
AttributeError: 'GenericResource' object has no attribute 'workspaceid'

01.train-within-notebook.pynb: error on Webservice.deploy_from_model()

I've successfully executed the 00.configuration.ipynb notebook, but when I try to execute the 01.train-within-notebook.ipynb, I get an error message when I'm trying to execute the following cell:

%%time
# this will take 5-10 minutes to finish
# you can also use "az container list" command to find the ACI being deployed
service = Webservice.deploy_from_model(name='myacisvcwbapsv',
                                       deployment_config=aciconfig,
                                       models=[model],
                                       image_config=image_config,
                                       workspace=ws)

service.wait_for_deployment(show_output=True)

Output:

Creating image
Image creation operation finished for image acirandomname:1, operation "Failed"
Image creation failed with
StatusCode: 400
Message: Docker Build Failed
---------------------------------------------------------------------------
WebserviceException                       Traceback (most recent call last)
<timed exec> in <module>()

~/anaconda2/envs/aml36/lib/python3.6/site-packages/azureml/core/webservice/webservice.py in deploy_from_model(workspace, name, models, image_config, deployment_config, deployment_target)
    251         image.wait_for_creation()
    252         if image.creation_state != 'Succeeded':
--> 253             raise WebserviceException('Error occurred creating image {} for service.'.format(image.id))
    254         return Webservice.deploy_from_image(workspace, name, image, deployment_config, deployment_target)
    255 

WebserviceException: Error occurred creating image acirandomname:1 for service.

I've already tried to execute this code on Azure Notebooks and also locally (I've created a new conda env for this deployment and installed all the AML dependencies).

The same error appears on both of these environments.

permissions issue

I've tried to get Workspace from config which I do have access to, but it fails with the following error:
SDK version: 0.1.80
Found the config file in: C:\Users\gubert\Repos\Gimmonix\HotelMappingAI\aml_config\config.json
get_workspace error using subscription_id=xxxxxxxxxxxxxxxxxxxxxxx, resource_group_name=xxxxxxxxxxxx, workspace_name=gmx-ml-mapping
Traceback (most recent call last):
File "C:\Users\gubert.azureml\envs\myenv\lib\site-packages\azureml_project_commands.py", line 320, in get_workspace
workspace_name)
File "C:\Users\gubert.azureml\envs\myenv\lib\site-packages\azureml_base_sdk_common\workspace\operations\workspaces_operations.py", line 78, in get
raise models.ErrorResponseWrapperException(self._deserialize, response)
azureml._base_sdk_common.workspace.models.error_response_wrapper.ErrorResponseWrapperException: Operation returned an invalid status code 'Forbidden'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "c:\Users\gubert.vscode\extensions\ms-python.python-2018.10.1\pythonFiles\experimental\ptvsd_launcher.py", line 38, in <module>
main(sys.argv)
File "c:\Users\gubert.vscode\extensions\ms-python.python-2018.10.1\pythonFiles\experimental\ptvsd\ptvsd_main_.py", line 265, in main
wait=args.wait)
File "c:\Users\gubert.vscode\extensions\ms-python.python-2018.10.1\pythonFiles\experimental\ptvsd\ptvsd_main_.py", line 256, in handle_args
run_main(addr, name, kind, *extra, **kwargs)
File "c:\Users\gubert.vscode\extensions\ms-python.python-2018.10.1\pythonFiles\experimental\ptvsd\ptvsd_local.py", line 52, in run_main
runner(addr, name, kind == 'module', *extra, **kwargs)
File "c:\Users\gubert.vscode\extensions\ms-python.python-2018.10.1\pythonFiles\experimental\ptvsd\ptvsd\runner.py", line 32, in run
set_trace=False)
File "c:\Users\gubert.vscode\extensions\ms-python.python-2018.10.1\pythonFiles\experimental\ptvsd\ptvsd_vendored\pydevd\pydevd.py", line 1283, in run
return self._exec(is_module, entry_point_fn, module_name, file, globals, locals)
File "c:\Users\gubert.vscode\extensions\ms-python.python-2018.10.1\pythonFiles\experimental\ptvsd\ptvsd_vendored\pydevd\pydevd.py", line 1290, in _exec
pydev_imports.execfile(file, globals, locals) # execute the script
File "c:\Users\gubert.vscode\extensions\ms-python.python-2018.10.1\pythonFiles\experimental\ptvsd\ptvsd_vendored\pydevd_pydev_imps_pydev_execfile.py", line 25,
in execfile
exec(compile(contents+"\n", file, 'exec'), glob, loc)
File "c:\Users\gubert\Repos\Gimmonix\HotelMappingAI\test.py", line 8, in <module>
ws = Workspace.from_config()
File "C:\Users\gubert.azureml\envs\myenv\lib\site-packages\azureml\core\workspace.py", line 153, in from_config
auth=auth)
File "C:\Users\gubert.azureml\envs\myenv\lib\site-packages\azureml\core\workspace.py", line 86, in init
auto_rest_workspace = _commands.get_workspace(auth, subscription_id, resource_group, workspace_name)
File "C:\Users\gubert.azureml\envs\myenv\lib\site-packages\azureml_project_commands.py", line 326, in get_workspace
resource_error_handling(response_exception, WORKSPACE)
File "C:\Users\gubert.azureml\envs\myenv\lib\site-packages\azureml_base_sdk_common\common.py", line 270, in resource_error_handling
raise ProjectSystemException(response_message)
azureml.exceptions._azureml_exception.ProjectSystemException: {
"error_details": {
"error": {
"code": "AuthorizationFailed",
"message": "The client '[email protected]' with object id 'xxxxxxxxxxxxx' does not have authorization to perform action 'Microsoft.MachineLearningServices/workspaces/read' over scope '/subscriptions/xxxxxxxxxxxxxx/resourceGroups/CarsolizeCloud - Test Global/providers/Microsoft.MachineLearningServices/workspaces/gmx-ml-mapping'."
}
},
"status_code": 403,
"url": "https://management.azure.com/subscriptions/xxxxxxxxxxxxx/resourceGroups/CarsolizeCloud%20-%20Test%20Global/providers/Microsoft.MachineLearningServices/workspaces/gmx-ml-mapping?api-version=2018-03-01-preview"
}

---

Document Details

⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

• ID: 4f93a584-8199-d04b-788e-53ce99ef483a
• Version Independent ID: c1ee7a99-6d4b-3763-446a-912077183ccb
• Content: azureml.core.workspace.Workspace class
• Content Source: AzureML/docs-ref-autogen/azureml-core/azureml.core.workspace.Workspace.yml
• Service: machine-learning
• GitHub Login: @j-martens
• Microsoft Alias: jmartens

AttributeError: '_AutoMLRegressor' object has no attribute 'logger'

while running automl on regression problem, for two of the initial models:

ITERATION PIPELINE DURATION METRIC BEST
0 SparseNormalizer ElasticNet 0:00:16.517861 0.149 0.149
1
Exception ignored in: <bound method _AutoMLBase.del of <azureml.train.automl._automlregressor._AutoMLRegressor object at 0x7f78d690a860>>
Traceback (most recent call last):
File "/home/nbuser/anaconda3_501/lib/python3.6/site-packages/azureml/train/automl/_automlbase.py", line 180, in del
if self.logger is not None:
AttributeError: '_AutoMLRegressor' object has no attribute 'logger'
SparseNormalizer ExtraTreesRegressor 0:00:23.785131 0.150 0.149
2

Exception ignored in: <bound method _AutoMLBase.del of <azureml.train.automl._automlregressor._AutoMLRegressor object at 0x7f78d690a860>>
Traceback (most recent call last):
File "/home/nbuser/anaconda3_501/lib/python3.6/site-packages/azureml/train/automl/_automlbase.py", line 180, in del
if self.logger is not None:
AttributeError: '_AutoMLRegressor' object has no attribute 'logger'

StandardScalerWrapper DecisionTreeRegr0:00:15.952069 0.159 0.149
3 PCA ElasticNet 0:00:16.759112 0.150 0.149

Error submitting jobs using hyperdrive

Here is the error message:

Traceback (most recent call last):
File "/mnt/batch/tasks/shared/LS_root/jobs/wstorontbatchai_dpvnbyza/azureml/nl2sq_1543173288196_1/mounts/azureml_project_share/azureml/container_control_script_nl2sq_1543173288196_1.py", line 195, in
project_fetcher.fetch_project_snapshot(options.snapshot)
File "/mnt/batch/tasks/shared/LS_root/jobs/wstorontbatchai_dpvnbyza/azureml/nl2sq_1543173288196_1/mounts/azureml_project_share/azureml/nl2sq_1543173288196_1/azureml-setup/project_fetcher.py", line 49, in fetch_project_snapshot
_download_tree(sas_tree, ["."])
File "/mnt/batch/tasks/shared/LS_root/jobs/wstorontbatchai_dpvnbyza/azureml/nl2sq_1543173288196_1/mounts/azureml_project_share/azureml/nl2sq_1543173288196_1/azureml-setup/project_fetcher.py", line 97, in _download_tree
_download_tree(child, path_stack)
File "/mnt/batch/tasks/shared/LS_root/jobs/wstorontbatchai_dpvnbyza/azureml/nl2sq_1543173288196_1/mounts/azureml_project_share/azureml/nl2sq_1543173288196_1/azureml-setup/project_fetcher.py", line 89, in _download_tree
fh.write(_read_response(response))
UnicodeEncodeError: 'ascii' codec can't encode characters in position 12403-12405: ordinal not in range(128)

Error in 01train-within.notebook

Hi,

this notebook gives me an error almost at the end when trying to feed the entire test set and calculate the errors (residual values)

this is what I'm running

# score the entire test set.
test_samples = json.dumps({'data': X_test.tolist()})

result = json.loads(service.run(input_data = test_samples))
residual = result - y_test

And this is the error

TypeError                                 Traceback (most recent call last)
<ipython-input-33-88699137f46a> in <module>()
      2 test_samples = json.dumps({'data': X_test.tolist()})
      3 
----> 4 result = json.loads(service.run(input_data = test_samples))
      5 residual = result - y_test

~/anaconda3_501/lib/python3.6/json/__init__.py in loads(s, encoding, cls, object_hook, parse_float, parse_int, parse_constant, object_pairs_hook, **kw)
    346         if not isinstance(s, (bytes, bytearray)):
    347             raise TypeError('the JSON object must be str, bytes or bytearray, '
--> 348                             'not {!r}'.format(s.__class__.__name__))
    349         s = s.decode(detect_encoding(s), 'surrogatepass')
    350 

TypeError: the JSON object must be str, bytes or bytearray, not 'list'

Definition of "name"

Under deploy, deploy_from_model and deploy_from_image, you define "name" as "The name to give the deployed workspace". I think it should be "The name to give the deployed service."

---

Document Details

⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

• ID: bf44bed6-8c8a-cc6b-4981-62011da8db48
• Version Independent ID: 0faf4700-36b9-a930-27c1-27d3085653af
• Content: azureml.core.webservice.Webservice class
• Content Source: AzureML/docs-ref-autogen/azureml-core/azureml.core.webservice.Webservice(class).yml
• Service: machine-learning
• GitHub Login: @j-martens
• Microsoft Alias: jmartens

Test 1

Test

---

Document Details

⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

• ID: d0be5f00-b267-a544-5e1c-36fa254f7801
• Version Independent ID: b8b3eec3-8cf4-dc39-a4e3-f2ed5fa3590d
• Content: Azure Machine Learning SDK for Python
• Content Source: AzureML/docs-ref-conceptual/intro.md
• Service: machine-learning
• GitHub Login: @rastala
• Microsoft Alias: roastala

Cannot create a new Azure resource group and a new workspace.

This seems to be a simple question, it takes me long time.
Could anyone help me?

I try to create a new Azure resource group and a new workspace as below manual.
https://docs.microsoft.com/en-us/azure/machine-learning/service/quickstart-create-workspace-with-python

Python code

ws = Workspace.create(name='myaml',
                      subscription_id='xxxxxx-xxxx-xxxx-8ea0-991809afd024'
                      resource_group='Test',
                      create_resource_group=True,
                      location='Japan East' # or other supported Azure region
                     )

There is an error below:

File "", line 3
resource_group='Test',
^
SyntaxError: invalid syntax

What is wrong with above code?

Failed Image Creation: Unable to use Docker file to install ODBC driver

Migrating from Azure ML Experimentation. I added docker-file= argument to install MS SQL ODBC driver to the container. It worked fine on experimentation account but I got a 400 error stating Main file or dependency file failed to download.

Any idea how I can get the ODBC driver installed? Thanks!

Dockerfile:

# apt-get and system utilities
RUN apt-get update && apt-get install -y \
    curl apt-utils apt-transport-https debconf-utils gcc build-essential g++-5\
    && rm -rf /var/lib/apt/lists/*

# adding custom MS repository
RUN curl https://packages.microsoft.com/keys/microsoft.asc | apt-key add -
RUN curl https://packages.microsoft.com/config/ubuntu/16.04/prod.list > /etc/apt/sources.list.d/mssql-release.list

# install SQL Server drivers
RUN apt-get update && ACCEPT_EULA=Y apt-get install -y msodbcsql unixodbc-dev

# install SQL Server tools
RUN apt-get update && ACCEPT_EULA=Y apt-get install -y mssql-tools
RUN echo 'export PATH="$PATH:/opt/mssql-tools/bin"' >> ~/.bashrc
RUN /bin/bash -c "source ~/.bashrc"

# python libraries
RUN apt-get update && apt-get install -y \
    python-pip python-dev python-setuptools \
    --no-install-recommends \
    && rm -rf /var/lib/apt/lists/*

# install necessary locales
RUN apt-get update && apt-get install -y locales \
    && echo "en_US.UTF-8 UTF-8" > /etc/locale.gen \
    && locale-gen

# install additional utilities
RUN apt-get update && apt-get install gettext nano vim -y

Azure Notebook Code:

# Create image
from azureml.core.webservice import Webservice
from azureml.core.image import ContainerImage

image_config = ContainerImage.image_configuration(execution_script = "score.py", 
                                    runtime = "python",
                                    conda_file = "conda_dependencies.yml",
                                    docker_file= "Dockerfile",
                                    schema_file= "service_schema.json",
                                    tags = {"product": "xxx", "type": "recommendation"})

image = ContainerImage.create(name = "xxx-image",
                                # this is the model object
                                models = "model.pkl",
                                image_config = image_config,
                                workspace = ws)

# Wait for the create process to complete
image.wait_for_creation(show_output = True)

Output:

Creating image
Running............
FailedImage creation operation finished for image mmm-main-image:6, operation "Failed"
Image creation failed with
StatusCode: 400
Message: Main file or dependency file failed to download, after multiple attempts

Test 2

test

---

Document Details

⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

• ID: b327d029-fb4f-a602-5b18-0bae056f17da
• Version Independent ID: b91725d5-eb70-a741-ed26-19e394330a24
• Content: azureml.telemetry.logging_handler.AppInsightsLoggingHandler class
• Content Source: AzureML/docs-ref-autogen/azureml-telemetry/azureml.telemetry.logging_handler.AppInsightsLoggingHandler.yml
• Service: machine-learning
• GitHub Login: @j-martens
• Microsoft Alias: jmartens

No classifier_input defined in project-brainwave-quickstart.ipynb

In

model_def.pipeline.append(TensorflowStage(sess, classifier_input, classifier_output))

where is the definition of classifier_input ?

03.Build_model_runHistory.ipynb auth variable not working

CMD 6 has a ##TESTONLY comment, but that code block isn't working causing subsequent code referencing that arg to fail. This should be cleaned up, either with more descriptive comment or so, ideally, it works straight away.

03.auto-train-models automlconfig sample has non-optimal settings

While running the tutorial 03.auto-train-models.ipynb I was first confused because of the dataset selection which had conflicting implementation and comment (see #21). This was already merged, but I did some additional observations, which you may want to examine (and probably include related changes in the tutorial):

Running this notebook, it was hard for me to grasp what the Azure AutoML was actually doing, because the given AutoMLConfig has exit_score value 0.995, which is met already on the first pipeline - the execution will stop there, and it left me wondering what actually happened. It was only after I tuned the exit_score up by a notch that it realized to me how this thing actually works. I also had the benefit of peeking into the automl directory, which helped a lot - but for someone who starts with the tutorial, it may not be clear at the first glance.

To make the tutorial illustrate better how the Azure Automated Machine Learning feature works, I'd suggest the following changes:

1. Pick the dataset for training as in 01.auto-ml-classification.ipynb notebook (where 100 first samples on the set are left out deliberately to use them for testing at the end of the experiment) and fix the comment to match this intent. If you use only the 100 first samples for training, none of the classification pipelines seem to reach very high accuracy, and the execution with 100 samples is not that much faster than with 1673 that it would make a huge difference.

2. Use a higher exit_score to indicate the use of AutoML to identify the optimal pipeline. I tested with 0.9985 which results in two iterations, perfect for demonstration and still runs relatively fast.

03.auto-train-models: warnings from AutoMLConfig

With SDK version 0.1.80, the call to AutoMLConfig in 03.auto-train-models generates warnings about deprecated arguments. This replacement code eliminates the warnings:

from azureml.train.automl import AutoMLConfig

##Local compute 
Automl_config = AutoMLConfig(task = 'classification',
                             primary_metric = 'AUC_weighted',
                             iteration_timeout_minutes = 200,
                             iterations = 20,
                             n_cross_validations = 3,
                             experiment_exit_score = 0.9985,
                             blacklist_models = ['kNN','LinearSVM'],
                             X = X_train,
                             y = y_train,
                             path=project_folder)

Unusual blob/files synchronisation issues in SDK

NB: not a notebooks question, but I'm not sure where the SDK is located to raise an issue.

Background

We're running tensorflow object detection (TOD). Since we don't want to rewrite the code, we did a nasty thing by outputting the events + checkpoints to the ./logs directory. This means we can use the Azure ML Tensorboard plugin (which looks for ./logs).

Problem

The events get synced fine when we run tensorboard (the Azure ML contrib). However, TOD also writes checkpoints out to the same directory, so tensorboard syncs all of these too. The initial problem we have was that when we went to use the synced checkpoints ('cos why not), the file names were associated to the incorrect file. For example, we'd expect something like

X.index
X.meta
X.data

However, we found the content of X.meta was actually that of X.index, etc.

Digging a bit further, we found that the associated fileshare (azureml/<experiment>/logs) was 'correct', and didn't have this issue:

whereas the files in the associated blob account (azureml/ExperimentRuns/<experiment>/logs) were incorrect - note that the size of the meta and index files has switched. (Ignore the tempstate file which I think is related to partial downloads or something.)

Since Tensorboard syncs using the RunWatcher etc. which uses the blobs not the file share (verified by printing the URI + filepath inside of RunWatcher), this is why our local files end up with incorrect names. We can't really provide a test case, but I'd start by looking for cases where you split off the extension (possibly while trying to get mimetype?).

On a side note - only the last five X are kept by TOD, and it overwrites as it goes. The file store correctly shows this, but not the blob account - i.e. the blob account doesn't appear to be being synced correctly to remove files that are deleted (when run on the container).

Broken link to DBC archve

The DBC archive for the Databricks examples (https://github.com/Azure/MachineLearningNotebooks/blob/master/databricks/Databricks_AMLSDK_github.dbc) is broken.

This is referred to in the GA documentation page: https://docs.microsoft.com/en-us/azure/machine-learning/service/how-to-configure-environment#azure-databricks

Trying to attach existing AKS Cluster

I am following through the example at https://github.com/Azure/MachineLearningNotebooks/blob/master/01.getting-started/11.production-deploy-to-aks/11.production-deploy-to-aks.ipynb and encountered an error when trying to attach my existing AKS cluster.

It should be noted that the first time I tried to connect I got a "You do not have enough cores. " error. So, I assume that it was able to at least query the cluster. I scaled up my AKS cluster to 6 cores and am trying to redeploy; however, now it's giving me a "Forbidden" error.

# Use the default configuration (can also provide parameters to customize)
resource_id = '/subscriptions/df8428d4-bc25-4601-b458-1c8533ceec0b/resourcegroups/aks-1-11-aks-rg/providers/Microsoft.ContainerService/managedClusters/aks-1-11-aks-rg'

create_name='aks-1-11-aks' 
# Create the cluster
aks_target = AksCompute.attach(workspace=ws, name=create_name, resource_id=resource_id)
# Wait for the operation to complete
aks_target.wait_for_completion(True)

Result:

Creating.
FailedProvisioning operation finished, operation "Failed"
Async operation failed with
StatusCode: 500
Message: An internal server error occurred. Please try again. If the problem persists, contact support
Compute object has provisioning state "Failed"
and provisioning errors: [{'error': {'code': 'InternalServerError', 'statusCode': 500, 'message': 'An internal server error occurred. Please try again. If the problem persists, contact support', 'details': [{'code': 'An internal server error occurred. Please try again. If the problem persists, contact support', 'message': "Operation returned an invalid status code 'Forbidden'"}]}}]

01.train-models.ipynb cannot import AmlCompute from Azure Notebooks

Running the 01.train-models.ipynb from Azure Notebooks and keep running into this error:

---

ImportError Traceback (most recent call last)
in
----> 1 from azureml.core.compute import AmlCompute
2 from azureml.core.compute import ComputeTarget
3 import os
3

ImportError: cannot import name 'AmlCompute'

ComputeTarget imports just fine. Got a workaround from a colleague by using BatchAiCompute

Need to change link in 03.auto-train-models

At the very bottom of the 0.3auto-train-models tutorial there is a link

how to configure settings for automatic training

that takes you to
https://www.microsoft.com/en-us/

this should be fixed

Lacking detail

It would be really helpful if this had some, you know, information. Please and thank you.

---

Document Details

⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

• ID: f82f79f7-e866-8e4b-9b9d-951c47a6e9d9
• Version Independent ID: b8257361-faf4-16a9-ab9c-0782bafeccd0
• Content: azureml.core.runconfig.DockerEnvironment class
• Content Source: AzureML/docs-ref-autogen/azureml-core/azureml.core.runconfig.DockerEnvironment.yml
• Service: machine-learning
• GitHub Login: @j-martens
• Microsoft Alias: jmartens

Errors on Azure Notebook

You might want to add a comment to include the following line at the top if running on Azure Notebooks:

!pip install azureml-contrib-brainwave

08.auto-ml-remote-execution-with-text-file-on-DSVM.ipynb missing Modules

are missing automl module and pandas_ml Module

The constructor is not documented.

It would be useful to understand what the parameters in the constructor are. Could you add some documentation for them?

---

Asiakirjan tiedot

⚠ Älä muokkaa tätä osiota. Sitä tarvitaan kohteessa docs.microsoft.com GitHub-ongelmien linkityksessä.

• ID: d11bca2c-939e-2125-7dae-aee7d37c8844
• Version Independent ID: f362dc68-c243-df06-976d-9cc1eb5d427c
• Content: azureml.core.Workspace class
• Content Source: AzureML/docs-ref-autogen/azureml-core/azureml.core.Workspace(class).yml
• Service: machine-learning
• GitHub Login: @j-martens
• Microsoft Alias: jmartens

Difficulty executing automl_setup.cmd

I'm trying to use this but am having difficulty getting off the ground. I made a new folder, copied the "automl_setup.cmd" file, located the folder in Command Prompt, and ran "automl_setup," as instructed. I got the error The syntax of the command is incorect. I tried opening the file directly, which opens & closes a window very quickly. I then tried to simply copy & run the 00.configuration.ipynb file and open it with Jupyter but it gave me a different error:
C:\[path]\00.configuration.ipynb NotJSONError("Notebook does not appear to be JSON: '\\n\\n\\n\\n\\n\\n<!DOCTYPE html>\\n<html lang...")

Any help would be greatly appreciated. I tried following the instructions in the readme as closely as I could but can't see what I'm doing wrong.

Phishing URL

I am unsure if this is the correct repo to ask this question, however, I am new to Azure ML and was interested in downloading Azure Machine Learning Studio.

I googled Azure ML and this is one of the top url's that appear https://studio.azureml.net/

I believe this is a phishing url because of the URL does not appear to match microsoft's domain. I would expect this URL to appear instead https://azure.microsoft.com/en-us/free/services/machine-learning/?v=18.45

It's not a bug, but I believe someone is trying to obtain user credentials by posing as Azure ML.

Wrong variable name in project-brainwave-custom-weights.ipynb

Second cell under "Deploy" heading in the notebook:

from azureml.core.model import Model
from azureml.core.image import Image
from azureml.core.webservice import Webservice
from azureml.contrib.brainwave import BrainwaveWebservice, BrainwaveImage

model_name = "catsanddogs-resnet50-model"
image_name = "catsanddogs-resnet50-image"
service_name = "modelbuild-service"

registered_model = Model.register(ws, service_def_path, model_name)
`
In the last line it should be `model_def_path`.

SDK installation on databricks

NOTE: Some packages like psutil upgrade libs that can cause a conflict, please install such packages by freezing lib version. Eg. "pstuil cryptography==1.5 pyopenssl==16.0.0 ipython=2.2.0" to avoid install error. This issue is related to Databricks and not related to AML SDK.

I tried installing psutil (there is a typo. See my PR #74) with the package dependencies as well as their versions, on my Databricks cluster, but "attaching" process just hangs there forever. I am wondering if I am doing the correct thing.

Appreciate any advice.

Workspace connection Issue

Hi,
I am able to connect to the workspace first time when I log in but when I am trying to connect to the workspace after 2-3 hours, I am getting this issue :

Please ensure you have network connection. Error detail: HTTPSConnectionPool(host='login.microsoftonline.com', port=443): Max retries exceeded with url: /common/oauth2/devicecode?api-version=1.0 (Caused by SSLError(SSLError("bad handshake: Error([('SSL routines', 'ssl3_get_server_certificate', 'certificate verify failed')],)",),))

May be it is because of any token expire issue.
Please reply if any other person has resolved this issue.

Regards,
Nikhil

Deploy automatically-trained model with builtin data preprocessing

Hi,

I deployed a automatically trained regressor with prepreprocessed data (AutoMLConfig 'preprocess' parameter set to True).

When I try to use the score api:

POST {scoring_uri}
{
	"data": ["Tuesday", -73.9377441406,40.7584152222,-73.9377593994,40.7583885193]
}

I get the following error:
"{\"error\": \"'some_column_name'\"}"

Is there a way to know how to format the input for the score api?

Errors in Tensorboard example notebooks

There are a few errors in the current Tensorboard example notebooks in the training folder.
I was able to fix these following the documentation pdf

Before any imports there should be a line: !pip install azureml-contrib-tensorboard
As this module is not installed by following the Installation and Configuration notebook. Also the link to the Installation and Configuration notebook is broken in this example.

Also in code cell 7 arguments_list = ["--log_dir", "./logs_dir"] should be arguments_list = ["--log_dir", "./logs"]
Things seemed to work after I made these changes.

All recipes in /training should have covered all scenario

All samples should cover uploading training data to datastore and writing output models to ./outputs. Otherwise there will be a few blind spots for users.

I'd be happy to help modify the existing samples to include those.

10. Register Model, Create Image and Deploy Service - prerequisite notebook

10.register-model-create-image-deploy-service.ipynb includes in the first cell: "IMPORTANT: This notebook requires you to first complete "01.SDK-101-Train-and-Deploy-to-ACI.ipynb" Notebook".

It's not clear what notebook that refers to. Suggest either adding a link or updating.