muenzpraeger / salesforce-einstein-platform-apex

This project is no longer actively maintained. The functionality and future maintenance is available via the Einstein Vision and Language Model Builder in the AppExchange.

salesforce-einstein-platform-apex

This repository showcases how to use the Salesforce Einstein Platform API using an Apex based wrapper.

Please check the product documentation for general information about what the Salesforce Einstein Platform API is, how to use it and when it'll be available for you.

The wrapper supersedes the old wrapper for the Salesforce Einstein Vision API. Besided breaking changes compared to the old wrapper this repo contains the v2 of the API (including image-multi-label, language intent and language sentiment).

See the included Playground in action.

Version history

Check out the full changelog.

Prerequisites

For using the wrapper you'll need to fulfill the following requirements:

• Access to a Salesforce org, i. e. a Developer Edition or a scratch org (you can signup here for free if you don't have one).
• An API account for Salesforce Einstein Platform.

Please find the detailed instructions for how to setup access to the Einstein Platform API here.

Installation

Salesforce DX - new scratch org

Clone the repo to your local file system.

git clone https://github.com/muenzpraeger/salesforce-einstein-platform-apex

run the orgInit.sh script

./orgInit

Salesforce DX - deploy to an org from your hub

Again thanks to Wade for creating this neat feature.

Salesforce DX - deploy into developer edition or production org

You can use the Salesforce CLI to deploy the source into a regular Salesforce org using the Metatdata API.

Authenticate against the deployment org

sfdx force:auth:web:login -a yourOrgAlias

Create an output directory for the to be converted source

mkdir mdapi

Convert the source from Salesforce DX format to Metatdata API format

sfdx force:source:convert -r force-app -d mdapi

Deploy the source

sfdx force:mdapi:deploy -d mdapi -u yourOrgAlias

Configuration

After you've added the wrapper files two steps are required:

• Set the value for Einstein EMail in Custom Settings => Einstein Settings (via the Manage button) for that org to the email address that you've used to sign up for Einstein Platform.
• Store the Einstein Platform file as File in the org. The name must be einstein_platform. If you want to use this in production make sure that the certificate isn't shared publicly. An option is for example to create and store a custom encrypted blob, and then decrypt via code.

If you went through my Trailhead project you likely went through that excercise already.

The installation adds a new Lightning App to your Salesforce org for the included Playground.

If you want to use this in production make sure that the certificate isn't shared publicly. Options are for example to create and store a custom encrypted blob, and then decrypt via code.

Usage examples

Creating a PredictionService

The foundation for everything is the PredictionService. As the communication with the API is based on a valid OAuth2 token (see MetaMind documentation) you can initiate a new PredictionService in the following way.

Einstein_PredictionService predictionService = new Einstein_PredictionService(Einstein_PredictionService.Types.IMAGE);

This creates a new prediction service for working with images. You can switch the type so that you can reuse it for other predictions.

service.setType(Einstein_PredictionService.Types.SENTIMENT);

Fetch all trained image datasets

Einstein_PredictionService service = new Einstein_PredictionService(Einstein_PredictionService.Types.IMAGE);
List<Einstein_Dataset> datasets = service.getDatasets();

Train an intent dataset

Einstein_PredictionService service = new Einstein_PredictionService(Einstein_PredictionService.Types.INTENT);
Einstein_Model model = service.trainDataset(datasetId, 'the dataset name', 0, 0, '';

Prediction

You can predict images either by sending Base64, uploading a Blob or a remote (publicly available!) URL. See this example how to validate a remote URL.

Einstein_PredictionResult result = service.predictImageUrl('GeneralImageClassifier', 'yourUrl', 5, '');

The prediction for intent or sentiment is similar. Like this example for intent.

Einstein_PredictionResult result = service.predictIntent('yourModelId', 'theText', 0, '');

Contribution

Feel free to contribute to this project via pull requests. Please read the contribution before you start working on something.

License

For licensing see the included license file.

salesforce-einstein-platform-apex's Issues

metrics and learning curve response serialization

Here's some of what I'm running into:

1. I can get back a response from all types
2. image learning curves are fine. Metrics are fine for image and all language models
3. text return epochResults as {} and that causes an apex exception FATAL_ERROR System.JSONException: Expected List<Einstein_EpochResult> but found { at [line:1, column:295]
4. Like we discussed, you'll probably have to make a learning curve type
5. model metrics (and probably LC) for image-multi-label are also broken in the parser
6. Then getModelMetrics and getModelLearningCurves in the Einstein_PredictionService probably need to be type-aware for their parsing issues (the other option would be to split them into type-specific methods, but I like that option less)
7. Once those are done, I'll update my front-end for model-type-awareness

API usage returning 'null'

When clicking on the Einstein Admin tab I see the following error:

I'm using the managed package version of the app, available here:
Package Name: Einstein Playground
Version: Spring 2019 / 1.16.0
First Release: 4/9/2019
Latest Release: 4/10/2019

The Intro page points to this repo. Is the latest version of managed package in sync with 'master' branch of this repo?

Internal error occurs when email is not set in custom settings

If you do not set your email in the custom settings, an internal error occurs as soon as you navigate to the Playground or Einstein Usage tab.

The error does not tell anything about what's wrong. May I suggest that you reinstate my previous contribution with the configuration checks that were reporting config errors?

Source API version should be set to v41 (Winter '18)

https://developer.salesforce.com/docs/atlas.en-us.sfdx_dev.meta/sfdx_dev/sfdx_dev_ws_config.htm

Show spinner when image got classified

Currently no loading spinner is shown when the image prediction runs. As the process may take a couple of seconds (based on network) it's useful to show some loading indication.

Default certificate name

After install app I put to custom settings default name of the certificate. The app didn't find a certification

Need to move Lightning Error Handler component to correct path

Need to move Lightning Error handler aura component to below path.
salesforce-einstein-platform-apex/force-app/main/default/aura/

As while doing deployment using sf CLI, it gives error.

UI fixes before "GA"

Model switching error

Switching a model throws an error when no data is available (i. e. on image classification, sentiment etc.)

Reproduce

1. Open Playground
2. Select tab "Image Classification"
3. Select tab "2. Prediction"
4. Select any model (without having an image or url defined)

Empty model

For any of the models where pre-build models are available an empty model entry gets created (happens only when no custom models are available)

Create einsteinSetup cmd script

Currently there is only a Mac/*nix version for the automated org setup. A Windows script is needed too.

PEM not found in Future Method or Queueable Apex

For whatever reason the PEM file is not found when I run my code as a Future or Queueable Apex class. However, the file IS found when I run it as "anonymous execution"

Sandbox

Apex script unhandled exception by user/organization: 00539000005k15c/00D4C0000008m4E
Source organization: 00D390000003wR6 (null)
Failed to invoke future method 'public static void detectURL3(String)' on class 'EinsteinVisionHelper' for job id '7074C00000Twcnn'

caused by: Einstein_ConfigurationException: Could not retrieve the Einstein Platform certificate file: einstein_platform_playground

Class.Einstein_PredictionService.getAccessToken: line 965, column 1
Class.Einstein_PredictionService.<init>: line 34, column 1
Class.EinsteinVisionHelper.detectURL3: line 21, column 1
Caused by
Class.Einstein_PredictionService.getAccessToken: line 963, column 1
Class.Einstein_PredictionService.<init>: line 34, column 1
Class.EinsteinVisionHelper.detectURL3: line 21, column 1

Restrict file upload size for image classification

To avoid the need for a chunked file upload/saving a temporary file within the org a limit for the image classification upload needs to be in place.

Null exception from EinsteinDataset component

If a user clicks into the 'Models' tab before actually training a model, the component throws an unhandled exception.

https://github.com/muenzpraeger/salesforce-einstein-platform-apex/blob/master/force-app/main/default/aura/EinsteinDataset/EinsteinDatasetHelper.js
Line 26: https://github.com/muenzpraeger/salesforce-einstein-platform-apex/blob/master/force-app/main/default/aura/EinsteinDataset/EinsteinDatasetHelper.js

See fix:
https://github.com/muenzpraeger/salesforce-einstein-platform-apex/commit/6cfc5e827315377cf0084d95016c029940834b10

No defaultusername org found.

Hi,

I have followed the steps to install the app into a fresh new scratch org but the following error is triggered:

ERROR: No defaultusername org found.

Try this:
Run the "sfdx force:auth" commands with --setdefaultusername to connect to an org and set it as your default org.
Run "force:org:create" with --setdefaultusername to create a scratch org and set it as your default org.
Run "sfdx force:config:set defaultusername=" to set your default username.

According the salesforce documentation. I can set a user but because this step isn't referenced, I wanted to check if you thing something is missing on your explanations.

Regards,

More descriptive errors

Often, when an exception occurs, an alert is sent that says "An internal server error has occurred". For example, if one does not have access to the pem file or they put in the wrong email address.

Perhaps we could instead surface the actual error description, that way administrators don't need to sort through the logs.

Test cases are failing if the org has a namespace defined.

Three of the test cases in Test_Einstein are failing if the org running the tests has a namespace defined.

1. Test_Einstein.testOtherMethods
2. Test_Einstein.trainDataset
3. Test_Einstein.createExamplesFromUrl

All are asserting that the Exception type will name is of an expected value. However, the presence of a namespace in the org alters the full type name.

You could change the assertions to use instanceof.

        try {
            model = service.trainDataset(0, 'Beach and Mountain Model', 0, 0, '');
        } catch (Exception e) {
            System.debug('Error is: ' + e.getTypeName());
            System.assert(e instanceof Einstein_NoValuesException);
        }

Another alternative would be to only catch the expected exception type and fail anything else.

        try {
            model = service.trainDataset(0, 'Beach and Mountain Model', 0, 0, '');
            System.assert(false, 'Exception expected');
        } catch (Einstein_NoValuesException e) {
            System.debug('Error is: ' + e.getTypeName());
        }

Timeout for HTTP needs to be enhanced

The default timeout for HTTP callouts of 10 seconds may not be enough for some Einstein Vision predictions. For this the timeout needs to be increased to 30 seconds.

Object detection fails with error "Bad request: modelType not supported for this endpoint"

I used the standard Alpine dataset and the sample picture from Einstein.ai
https://einstein.ai/products/custom-detection

??

Cache API Token

Every time a request is made, the implementation actually does two calls; one to get an access token and a second to do the actual request. One could temporarily cache the token, say for an hour, reducing the volume of outbound API requests an improving performance.

Null exception from ImageClassification component

If a user tries to interact with the Prediction subtab & models picklist before uploading a file image, the component throws an unhandled null exception.

Issue:
https://github.com/muenzpraeger/salesforce-einstein-platform-apex/blob/master/force-app/main/default/aura/EinsteinImageClassification/EinsteinImageClassificationController.js
Begins on Line 4:
if (files && files.length > 0)

See fix (controller.js markup changes):
https://github.com/muenzpraeger/salesforce-einstein-platform-apex/commit/429298f8b34b52c733e95b02b491e1c06f375664#diff-83f5f29d4ef7e13f350f2c78b816dbb6

Fetch all models when datasets are loaded

When the datasets of a specific type are loaded (i. e. all image classifications) the associated models should be loaded directly to pre-populate the prediction dropdown.

Installation error : pad block corrupted

Hello,
during the installation process I got this error related to "pad blocked corrupted"

1. My domain is active and published
2. Browser Crome
3. Environement : Dev Sandbox

Error when deploying Winter 18 Scratch Org

We are getting the below error when pushing the code to scratch org with Winter 18:

ERROR: Cannot read property 'getRepresentation' of undefined.

Stacktrace in the log:

"[ false,\n '{"message":"Cannot read property \'getRepresentation\' of undefined","status":1,"stack":"TypeError: Cannot read property \'getRepresentation\' of undefined\\n at AggregateSourceElement.composeMetadata (/Users/sfdx/.local/share/sfdx/client/node_modules/salesforce-alm/dist/lib/aggregateSourceElement.js:395:48)\\n at AggregateSourceElement.getFilePathTranslations (/Users/sfdx/.local/share/sfdx/client/node_modules/salesforce-alm/dist/lib/aggregateSourceElement.js:353:35)\\n at Promise.map.element (/Users/sfdx/.local/share/sfdx/client/node_modules/salesforce-alm/dist/lib/sourceConvertApi.js:114:42)\\n at tryCatcher (/Users/sfdx/.local/share/sfdx/client/node_modules/bluebird/js/release/util.js:16:23)\\n at MappingPromiseArray._promiseFulfilled (/Users/sfdx/.local/share/sfdx/client/node_modules/bluebird/js/release/map.js:61:38)\\n at MappingPromiseArray.PromiseArray._iterate (/Users/sfdx/.local/share/sfdx/client/node_modules/bluebird/js/release/promise_array.js:113:31)\\n at MappingPromiseArray.init (/Users/sfdx/.local/share/sfdx/client/node_modules/bluebird/js/release/promise_array.js:77:10)\\n at MappingPromiseArray._asyncInit (/Users/sfdx/.local/share/sfdx/client/node_modules/bluebird/js/release/map.js:30:10)\\n at Async._drainQueue (/Users/sfdx/.local/share/sfdx/client/node_modules/bluebird/js/release/async.js:143:12)\\n at Async._drainQueues (/Users/sfdx/.local/share/sfdx/client/node_modules/bluebird/js/release/async.js:148:10)\\n at Immediate.Async.drainQueues (/Users/sfdx/.local/share/sfdx/client/node_modules/bluebird/js/release/async.js:17:14)\\n at runCallback (timers.js:781:20)\\n at tryOnImmediate (timers.js:743:5)\\n at processImmediate [as _immediateCallback] (timers.js:714:5)","name":"TypeError","warnings":[]}' ]"

Order dataset by last update date (descending)

Order dataset view by last update date (descending)

Setup error message isn't displayed

The error message when the setup isn't performed correctly isn't displayed because of a missing negation.

salesforce-einstein-platform-apex/force-app/main/default/aura/EinsteinPlaygroundIntroduction/EinsteinPlaygroundIntroduction.cmp

Line 16 in 90a31fd

<aura:if isTrue="{!v.setupComplete}">

FYI @zaynelt

Add setup UI to playground

The setup experience should be simplified. Currently a user has multiple setup steps:

• Go go to Setup => Custom Settings and configure the email address.
• Go to files and upload the certificate file

Both steps should be added as a simple configuration UI in the playground itself, so that people have a faster start.

Update readme with certificate setup instructions

You probably want to update the configuration section of the readme with some certificate setup instructions.

einstein_platform file versions support

I made a mistake when first uploading the key file. I then uploaded the right one as a new version.
It didn't work as the code doesn't look for the latest version of a file.

A new query filter should be added to select the latest version of the einstein_platform file.

An Internal Server has occured.

All configuration is done in SF org. When I accessing the Image Classification tab, that time I got the following error:

please help me to resolve this error.

Add UI example for using the wrapper

Error sending image feedback using createFeedbackImageExample

I'm getting the following error when sending jpg image feedback using the createFeedbackImageExample method: "File content must be provided in one of the supported file formats: [zip, octet-stream, jpeg, png, binary]"

Einstein platform expects jpeg (as per the error message) rather than jpg. So If I change the mimeType from 'image/jpg' to 'image/jpeg' on line 177 in Einstein_HttpBodyPart.cls it works.

Request you to please fix it.

Run image prediction after image been selected

To reduce the clicks the image prediction should kick off after the user selected an image.

Feature request: project packaging

Could you consider packaging your project in order to make it easy to integrate into other projects as a dependency?

Right now, we have to deploy it to a scratch org and export/import it as metadata but this is time consuming and does not support versioning.
I would love to be able to install a specific version in a project with just one DX command:
sfdx force:package:install -i 04xxxxxxxx

Another extra nice-to-have feature would be to separate the core backend wrapper from the Playground UI to lighten up the project.

Pretty please 😍

Can't run an image classification prediction from a URL

When trying to run an image classification prediction from a URL, the following Lightning error occurs:

Uncaught Action failed: c:EinsteinPredictionArea$controller$predictURL [Cannot read property 'length' of undefined]

Note, when uploading the file and running the prediction, everything works as expected.

Null pointer when change model typ in ptredictoin tab

Null pointer when change model type in prediction tab
Exception

Steps to reproduce

how to store the private key in Certificate Management

The Playground Intro tab says, "Store the certificate as certificate in the Certificate Management section of your org". The private key file from Einstein.ai is in PEM format, and Certificate Management only takes JKS for import. So I set about to do a conversion following this:
https://docs.oracle.com/cd/E35976_01/server.740/es_admin/src/tadm_ssl_convert_pem_to_jks.html
and got stuck on the step of importing CA cert - I don't really have one. Am I doing this wrong?

Unable to install in the developer edition 12 errors

I am unable to deploy to a developer edition org . The org is basically a Trailhead playground . I tried in multiple orgs all have the same failure. Request your help.

=== Component Failures [12]
TYPE FILE NAME PROBLEM
───── ─────────────────────────────────────────────────────────────────────────────────────── ────────────────────────────────── ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Error mdapi/profiles/Admin.profile Admin Unknown user permission: CreateWorkBadgeDefinition
Error mdapi/aura/EinsteinDataAndTraining/EinsteinDataAndTraining.cmp EinsteinDataAndTraining No COMPONENT named markup://c:LightningErrorHandler found : [markup://c:EinsteinDataAndTraining, markup://c:EinsteinDataset]
Error mdapi/aura/EinsteinDataset/EinsteinDataset.cmp EinsteinDataset No COMPONENT named markup://c:LightningErrorHandler found : [markup://c:EinsteinDataset]
Error mdapi/aura/EinsteinDatasetCreation/EinsteinDatasetCreation.cmp EinsteinDatasetCreation No COMPONENT named markup://c:LightningErrorHandler found : [markup://c:EinsteinDatasetCreation]
Error mdapi/aura/EinsteinGenericPlaygroundComponent/EinsteinGenericPlaygroundComponent.cmp EinsteinGenericPlaygroundComponent No COMPONENT named markup://c:LightningErrorHandler found : [markup://c:EinsteinGenericPlaygroundComponent, markup://c:EinsteinDataAndTraining, markup://c:EinsteinDataset]
Error mdapi/aura/EinsteinGenericPlaygroundComponent/EinsteinGenericPlaygroundComponent.design EinsteinGenericPlaygroundComponent No COMPONENT named markup://c:LightningErrorHandler found : [markup://c:EinsteinGenericPlaygroundComponent, markup://c:EinsteinGenericPlaygroundComponent, markup://c:EinsteinDataAndTraining, markup://c:EinsteinDataset]
Error mdapi/aura/EinsteinLanguageModelBuilder/EinsteinLanguageModelBuilder.cmp EinsteinLanguageModelBuilder No COMPONENT named markup://c:LightningErrorHandler found : [markup://c:EinsteinLanguageModelBuilder]
Error mdapi/aura/einsteinModelMetrics/einsteinModelMetrics.cmp einsteinModelMetrics No COMPONENT named markup://c:LightningErrorHandler found : [markup://c:einsteinModelMetrics]
Error mdapi/aura/EinsteinModelSelect/EinsteinModelSelect.cmp EinsteinModelSelect No COMPONENT named markup://c:LightningErrorHandler found : [markup://c:EinsteinModelSelect]
Error mdapi/aura/EinsteinPlayground/EinsteinPlayground.cmp EinsteinPlayground No COMPONENT named markup://c:LightningErrorHandler found : [markup://c:EinsteinPlayground, markup://c:EinsteinGenericPlaygroundComponent, markup://c:EinsteinDataAndTraining, markup://c:EinsteinDataset]
Error mdapi/aura/EinsteinPredictionArea/EinsteinPredictionArea.cmp EinsteinPredictionArea No COMPONENT named markup://c:LightningErrorHandler found : [markup://c:EinsteinPredictionArea]
Error mdapi/aura/EinsteinUsage/EinsteinUsage.cmp EinsteinUsage No COMPONENT named markup://c:LightningErrorHandler found : [markup://c:EinsteinUsage]

Increase BodyPart subclass code coverage

Make full use of getters/setters in the object creation body to increase code coverage for those classes.

Put the highest epoch onto the dataset's labelSummary

Put the best epoch onto the dataset's Summary.
On screen number 1,2 you will see mismatch between Summary f1 and Model Epoch 15 f1.
1.
2.
Here is best apoch
3.

Einstein_Settings__c missing after deploying to new scratch org.

I used the "Deploy to SFDX" button to spin up a new scratch org. It seemed to work, but I'm missing the Einstein_EMail__c hierarchy custom setting in the resulting org.
Is there something off with the Einstein_Settings__c?

I don't think the Einstein_Settings__c.object-meta.xml should be in a sub directory. Maybe it got skipped in the scratch org create?

Document einsteinSetup bash script

The included bash script to automatically set up the email address and the certificate file needs to be documented.

Add version changelog

To enhance usability of the repo a changelog should be added.

Error messages are not returned to Lighting UI

Using Aura exceptions bubbling up error messages to the Playground UI for better usability is needed.

Align table contents in dataset table

The headers and content row in the dataset are not vertically aligned. A minor CSS update is needed.

Don't have password of scratch org created using "SFDC Deploy"

I clicked on "Deploy to SFDX", this created the scratch org and given the link to login to scratch org.
Now after one day i am trying to login to scratch org with given link, but can't.
Trying to reset the password , then it ask "What is your pet's name?"

Please help.

System.NoDataFoundException: Data Not Available

I have created one Dev HUb org and deployed this code in it. I have created Einstein API account and downloaded my private key and done required changes in custom setting. Set org wide email id.

Now, I am going to Einstein Platform and started running the same but I am getting below error message:

System.NoDataFoundException: Data Not Available: The data you were trying to access could not be found. It may be due to another user deleting the data or a system error. If you know the data is not deleted but cannot access it, please look at our support page.

Can you please help me in resolving the same?
Is there any particular step here which i am missing?
Also, if I generate token for my private key, how to apply that new one in this app? Or will it be programmed in such a way that it will pick automatically?

Please help asap on this.

Thanking in advance.
Best regards

Unable to parse JSON model response where trainParams is defined

The JSON parsing fell over with the following exception when I was attempting to retrain a Datasets model in place.

System.JSONException: Illegal value for primitive
Class.System.JSONParser.readValueAs: line 117, column 1
Class.DFB.Einstein_PredictionService.retrainDataset: line 453, column 1

The JSON that came back in the response body:

{
    "datasetId": 1010273,
    "datasetVersionId": 0,
    "name": "Training1503306423791",
    "status": "QUEUED",
    "progress": 0,
    "createdAt": "2017-08-26T09:17:14.000+0000",
    "updatedAt": "2017-08-26T09:17:14.000+0000",
    "learningRate": 0.0,
    "epochs": 0,
    "queuePosition": 3,
    "object": "training",
    "modelId": "HAPDU3OCJB2WRFKZXVQXYEO4JA",
    "trainParams": {
        "withFeedback": true
    },
    "trainStats": null,
    "modelType": "image"
}

I suspect it is the trainParams that is causing the parsing problem.

Simplified parsing example:

string responseBody = '{    "datasetId": 1010273,    "datasetVersionId": 0,    "name": "Training1503306423791",    "status": "QUEUED",    "progress": 0,    "createdAt": "2017-08-26T09:17:14.000+0000",  "updatedAt": "2017-08-26T09:17:14.000+0000",    "learningRate": 0.0,    "epochs": 0,    "queuePosition": 3,    "object": "training",    "modelId": "HAPDU3OCJB2WRFKZXVQXYEO4JA",    "trainParams": {        "withFeedback": true    },    "trainStats": null,    "modelType": "image"}';
JSONParser parser = JSON.createParser(responseBody);
Einstein_Model model = (Einstein_Model)parser.readValueAs(Einstein_Model.class);

I suspect it needs a dedicated Einstein_TrainParams class. Something like:

public class Einstein_TrainParams {
    @AuraEnabled
    public Decimal trainSplitRatio {get;set;}
    @AuraEnabled
	public Boolean withFeedback {get;set;}
    @AuraEnabled
    public Long withGlobalDatasetId {get;set;}
}