palantir / palantir-r-sdk

Palantir R SDK

Palantir R SDK

Setup

Install the library from CRAN:

install.packages("foundry")

Alternatively, install the latest development version from Github:

install.packages("remotes")
remotes::install_github("https://github.com/palantir/palantir-r-sdk", ref = "0.13.0")

Configuration

Configuration for hostname and an authentication token can be provided using options:

• foundry.hostname is the hostname of your instance e.g. example.palantirfoundry.com
• foundry.token is a token acquired from the Tokens section of Foundry Settings

Alternatively, you can set the environment variables FOUNDRY_HOSTNAME and FOUNDRY_TOKEN.

Configuration will be loaded in the following order: options if present, otherwise environment variables.

Authentication tokens serve as a private password and allows a connection to Foundry data. Keep your token secret and do not share it with anyone else. Do not add a token to a source controlled or shared file.

Extra configuration options

• foundry.config_dir is the directory where configuration files are stored, defaults to ~/.foundry
• foundry.requests.timeout is the timeout of HTTPS requests made to Foundry, defaults to 150s

Aliases

To read datasets, aliases must first be registered. Create a YAML file called ~/.foundry/aliases.yml and define an alias for every dataset you wish to read or write to.

my_dataset:
    rid: ri.foundry.main.dataset.31388c03-2854-443e-b6cd-fe51c5908371
my_dataset_on_branch:
    rid: ri.foundry.main.dataset.5db9d133-6c87-4917-b11e-18b095ac4d30
    branch: develop

Examples

Read a tabular Foundry dataset into a data.frame or Apache Arrow Table

library(foundry)
df <- datasets.read_table("my_dataset")

Download raw files from a Dataset

all_dataset_files <- datasets.list_files("my_dataset")
downloaded_files <- datasets.download_files("my_dataset", all_dataset_files)

Write a data.frame or Apache Arrow Table to Foundry

datasets.write_table(df, "my_dataset")

Upload local files or folders to a Dataset

datasets.upload_files(file.path("~", "Downloads", "example"), "my_dataset")

palantir-r-sdk's Issues

Error retrieving object

Hi all,

first of all thank you for making this API for foundry available - us R users appreciate it :)

I was trying it but ran into an error which I can't figure out:

library("foundry")
Sys.setenv(FOUNDRY_HOSTNAME = "company.palantirfoundry.com")
df <- datasets.read_table("somedata")
Error in self$api_client$stop_for_status(response) : {
  "errorCode": "INTERNAL",
  "errorName": "Default:Internal",
  "errorInstanceId": "0160f807-21f7-4dd9-b0fa-78a3c9f79782",
  "parameters": {}
} (HTTP 500).

As far as I can tell the token is set in the system and the object exists. I have also set up the object yaml file in ~/.foundry. Any idea as what the source of the error might be?

---

Two somewhat related questions:

1. foundry.token, currently I have it has a system variable, how to set it for an interactive session?
2. Some of my datasets in Foundry are in json format. How to retrieve those?

---

Cheers

Function to Create a Dataset Rid

When writing a data.frame from the r environment to foundry you have to first create a rid and then alias it in the yaml file. I propose a function that would do this for you. Something like the following:

create_dataset <- function(name, parent_folder, write_to_yaml = TRUE){
    host_name <- Sys.getenv("FOUNDRY_HOSTNAME")
    token <- Sys.getenv("FOUNDRY_TOKEN")
    
    res <- httr::POST(
        url = glue::glue("https://{host_name}/api/v1/datasets"),
        httr::add_headers(
            "Authorization" = glue::glue("Bearer {token}")
        ),
        body = list(
            "name" = name,
            "parentFolderRid" = parent_folder
        ),
        encode = "json"
    )
    
    if (write_to_yaml){
        datasetRid <- httr::content(res)$rid
        write(
            glue::glue(
                "{name}:
                rid: {datasetRid}"
            ) ,
            "~/.foundry/aliases.yml",
            append = TRUE
        )
    }
    
    return (datasetRid)
}

Then it would be quick to create the table and upload data to it. Something like

create_dataset("mtcars", "{parentFolder}")
foundry::datasets.write_table(mtcars, "mtcars")