Submitting solutions direct from Pharo about pharo-smalltalk

[bencoman]

@macta @samWson, Could you review and test this...

To facilitate PHARO SUBMIT TESTING by Exercism staff and other curious punters unfamiliar with Pharo,
I've these instructions in detailed Quick Start form... no pre-requisites.

0. On the web site, join a fresh track (e.g. Python) in INDEPENDENT mode.
   Observe "hello world" status icon is "unlocked".
1. Download "Pharo standalone" for your platform from https://pharo.org/download (scroll down page).
2. Unzip and start Pharo.exe or pharo.
3. Left-click on background and choose ``Playground``` from the World menu.
4. Copy the following code into the Playground...

"CONFIGURE - LIVE EXERCISM WEB SITE"
apiToken := 'get it from https://exercism.io/my/settings'.
apiUrl := 'https://api.exercism.io/v1/solutions' asUrl.

"CONFIGURE - LOCAL DEV COPY EXERCISM WEBSITE"
apiToken := 'get it from http://lvh.me:3000/my/settings'.
apiUrl := 'http://lvh.me:3000/api/v1/solutions' asUrl.


"DOWNLOAD EXERCISE DATA"
client := ZnClient new.
client 
    url: apiUrl / 'latest' ;
    headerAt: 'Authorization' put: 'Bearer ' , apiToken ;
    queryAt: 'track_id'    put: 'python' ;
    queryAt: 'exercise_id' put: 'hello-world'.
response := client get.
solutionData := (STON fromString: response) at: 'solution'.
solutionData inspect. "also check website exercise icon has changed"


"SUBMIT EXERCISE"
iteration := 1 + (iteration ifNil: [ 0 ])."force change next line, otherwise Response(400 Bad Request)"
solutionEntity1 :=ZnByteArrayEntity bytes: 'MY SOLUTION #', iteration printString.
solutionEntity2 :=ZnByteArrayEntity bytes: 'SECOND PART OF MY SOLUTION.'.
solutionPart1 := ZnMimePart fieldName: 'files[]' fileName: 'mysolution.py' entity: solutionEntity1.
solutionPart2 := ZnMimePart fieldName: 'files[]' fileName: 'support.py' entity: solutionEntity2.
multiPartFormDataEntity := ZnMultiPartFormDataEntity new.
multiPartFormDataEntity 
    addPart: solutionPart1 ;
    addPart: solutionPart2. 
client := ZnClient new.
client
    url:  apiUrl / (solutionData at: 'id') ;
    headerAt: 'Authorization' put: 'Bearer ' , apiToken ;
    entity: multiPartFormDataEntity.
client patch.
client response inspect. "also check  website is updated"


"REVIEW CODE"
badMethod := (ZnMimePart class >> #fieldName:fileName:entity:).
badMethod browse.

"INSTALL FIX"
badCode := badMethod sourceCode.
goodCode := badCode copyReplaceAll: 'form-data;name=' with: 'form-data; name='.
ZnMimePart class compile: goodCode

"REVIEW FIX"
(DiffMorph from: badCode to: goodCode) openInWindow

Now "Do It" a bit at a time...

6. Select the two lines of the appropriate CONFIGURE, right-click and choose DoIt to evaluate them.
7. Evaluate "DOWNLOAD EXERCISE" section.
   A Dictionary of the JSON response should appear.
   Observe website status icon changed to "half/half in progress"
8. Evaluate "SUBMIT EXERCISE" section.
   A ZnRespone(500 Internal Server Error) should appear.
9. Evaluate "REVIEW CODE" section (and watch this window during next step).
10. Evaluate "INSTALL FIX" section.
11. Evaluate "SUBMIT EXERCISE" section again.
    A ZnResponse(201 Created application) should appear.
    Observe the website status icon changed to "thumbs up". Yay!"
12. Review the web site solution being updated every time the "DOWNLOAD EXERCISE" section is evaluated. Double-Yay!"
13. Evaluate "REVIEW FIX" section.

Note, while the "badCode" variable is used, its not clear where the fault lies per standards. We can fix it easily in our own application and maybe get it changed in Pharo distribution, but should the server be more permissive in its input?

[samWson]

I'm going to have a go at building this solution from the ExercismSubmit class stub. It's pushing my skills so I will probably make a few work in progress pull requests when I need help. Conveniently I've been building a lot of command pattern classes at work recently so hopefully I can make something that works.

[macta]

Go for it - I started to have a quick look, but didn’t get enough un-interupted time to get very far. I think you may need to push up some variables from the the other DownloadCommand that Ben refactored this too. I may have a look at some point (probably in 5 days when I have a long flight and can mull it over) - but lets see how far you get too, and we can compare notes. I think there is the question about storing some meta data stuff - that needs to be stored somewhere as a property on something - but don’t let that distract you yet. Tim

…

On 19 Sep 2018, at 00:02, Samuel ***@***.***> wrote: I'm going to have a go at building this solution from the ExercismSubmit class stub. It's pushing my skills so I will probably make a few work in progress pull requests when I need help. Conveniently I've been building a lot of command pattern classes at work recently so hopefully I can make something that works. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#96 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AFS919uP4679cNez8tk4N2-cqbREOc7gks5ucewAgaJpZM4WcJYt>.

[macta]

Sorry guys I’ve been off the grid (turns out the outback has limited 3G ;) That method rings a bell - TonelWriter is quite convoluted and I recall we had to do do some nasty overriding to coerce it to even do stuff. I will check as I filed a PR with Tonel.

[bencoman]

On Sat, 3 Nov 2018 at 14:54, Samuel ***@***.***> wrote: I didn't include the changes to TonelWriter in my last commit for that PR. They are summarised here <#110 (comment)>.

I did my last commit to that same PR. To get it working out of the box in a fresh image I added our required Tonel changes as an extension... 227172f#diff-e9dc21d4b8fe3f8ba613c5b452e4ed04R1 I've started discussion with Tonel guys about it... pharo-vcs/tonel#55 @bencoman <https://github.com/bencoman> since it looks like your going to

be busy for the time being

Yes, my day job it stretching me and I'm doing a lot of home reading to catch up my knowledge of hundreds of pages of standards. I'd be grateful if you can keep plugging away at it.

[bencoman]

First the download, for comparison.

C:> exercism --verbose download --track=python --exercise=hello-world

========================= BEGIN DumpRequest =========================
GET /v1/solutions/latest?exercise_id=hello-world&track_id=python HTTP/1.1
Host: api.exercism.io
Authorization: Bearer d4a3xxxx-secret-xxxxxxx59c56
Content-Type: application/json
User-Agent: github.com/exercism/cli v3.0.8 (windows/amd64)
========================= END DumpRequest =========================


========================= BEGIN DumpResponse =========================
HTTP/1.1 200 OK
Transfer-Encoding: chunked
Cache-Control: max-age=0, private, must-revalidate
Connection: keep-alive
Content-Type: application/json; charset=utf-8
Date: Thu, 06 Sep 2018 05:42:13 GMT
Etag: W/"e438ca3065bc3f2ff0a29cd1bf73f33e"
Referrer-Policy: strict-origin-when-cross-origin
Server: nginx/1.10.3 (Ubuntu)
Set-Cookie: site_context=normal; domain=.exercism.io; path=/
Set-Cookie: _exercism_session=5m2%2F6wlYMBWhcXhKNtlek7%2BPUo85TXxxlVzhaKYbIVJkOJ9GoKJAkuPTmPjs4MeJlu0kton8KdebLQDFUxwdvB4CR3yW%2BRlPgjiD6X41CpNudC%2B709enVtVovsb05l32cbmxpHo%2FQ%2Fu4YSn4nXMtY41yo31Hxh5mXOrbN5g2bK5MY23wm2iUzPV7Sqhf04S4OVEfgmh4eyx0s9K2efCkjW2D6H2f%2BG0ZMdZD6%2Fd9oWDYCBafEFNKz66%2B0LI2gMI1X1ofg1ndbg%3D%3D--b3qbJIbGa2LYal3x--j1zHyWqD9wgYRIrT5RqNZw%3D%3D; domain=.exercism.io; path=/; HttpOnly
X-Content-Type-Options: nosniff
X-Download-Options: noopen
X-Frame-Options: SAMEORIGIN
X-Permitted-Cross-Domain-Policies: none
X-Request-Id: 6d216682-8737-4b61-8a4c-cbd47dee59bf
X-Runtime: 0.102770
X-Xss-Protection: 1; mode=block
========================= END DumpResponse =========================


========================= BEGIN DumpRequest =========================
GET /v1/solutions/66f3b82a26c248d3934fcb11e708b25f/files/README.md HTTP/1.1
Host: api.exercism.io
Authorization: Bearer d4a3xxxx-secret-xxxxxxx59c56
Content-Type: application/json
User-Agent: github.com/exercism/cli v3.0.8 (windows/amd64)
========================= END DumpRequest =========================


========================= BEGIN DumpResponse =========================
HTTP/1.1 200 OK
Transfer-Encoding: chunked
Cache-Control: max-age=0, private, must-revalidate
Connection: keep-alive
Content-Type: text/plain; charset=utf-8
Date: Thu, 06 Sep 2018 05:42:14 GMT
Etag: W/"5aa9487af0280cceb18751f931b593be"
Referrer-Policy: strict-origin-when-cross-origin
Server: nginx/1.10.3 (Ubuntu)
Set-Cookie: site_context=normal; domain=.exercism.io; path=/
Set-Cookie: _exercism_session=%2F3RGG7GnSdRZVD%2FI7hhRYhTsF1FqwMHmlxmnuOd5M%2BDiGVoFZo%2B2vxZ2nSw4QIrNIFDsm7Xm3Z7K1lAkhP%2Bsx2tk%2BYaflHQPU5FxqXV8SNbnjLGbOBV62TfgxV9kHZhg48ZC9DFxjHHXXuH9fYCaHraw98LJ5T1wvXDQsdA6c72sx1G81Wm0XE31PrkibrPZQpz8EZKd16JvHvSPnf63RqFVk01msyAdoLLSFnqxMqfbTv4eUg8k0ews6FVD7EGdUCZ9WA%3D%3D--D1TA5%2BB%2F5dO7Fmfe--zjtS0ayiRM6ZryVvVjbjcA%3D%3D; domain=.exercism.io; path=/; HttpOnly
X-Content-Type-Options: nosniff
X-Download-Options: noopen
X-Frame-Options: SAMEORIGIN
X-Permitted-Cross-Domain-Policies: none
X-Request-Id: 17117035-cce2-4316-b331-be5194f8ff9e
X-Runtime: 0.069309
X-Xss-Protection: 1; mode=block
========================= END DumpResponse =========================


========================= BEGIN DumpRequest =========================
GET /v1/solutions/66f3b82a26c248d3934fcb11e708b25f/files/hello_world.py HTTP/1.1
Host: api.exercism.io
Authorization: Bearer d4a3xxxx-secret-xxxxxxx59c56
Content-Type: application/json
User-Agent: github.com/exercism/cli v3.0.8 (windows/amd64)
========================= END DumpRequest =========================


========================= BEGIN DumpResponse =========================
HTTP/1.1 200 OK
Transfer-Encoding: chunked
Cache-Control: max-age=0, private, must-revalidate
Connection: keep-alive
Content-Type: text/plain; charset=utf-8
Date: Thu, 06 Sep 2018 05:42:15 GMT
Etag: W/"e3163528c26697e825dd5eec25279a86"
Referrer-Policy: strict-origin-when-cross-origin
Server: nginx/1.10.3 (Ubuntu)
Set-Cookie: site_context=normal; domain=.exercism.io; path=/
Set-Cookie: _exercism_session=ynyH%2BeWpTBiSa%2FR1WOhvD9FI3CoAzg8sQq6ommUOcq27XbXhoPh8KrMwHp2R5HQ5Qewkry0BvKHZT3kOiXEAa%2BAD24lk%2FzqQCBv%2BggbahAqxEvZNmC5FLQUeyTUE12k2SxK8XZLyD%2FHJ%2F7OF0wfvz7FOINkktpglX9wOIvzniab07i7JBvzvdFkfXlIt2BZZqChjUH34BdFrUt6sKJkjLDI3T%2F2YPXXFm1w8IRW3i5x%2F96rWPEUEvFfgbWghBRAwBKcVe2K0XQsx--X4PCsgG8bqHOgpM3--ZirJWNTuYF0h8OXXy0KNKA%3D%3D; domain=.exercism.io; path=/; HttpOnly
X-Content-Type-Options: nosniff
X-Download-Options: noopen
X-Frame-Options: SAMEORIGIN
X-Permitted-Cross-Domain-Policies: none
X-Request-Id: f5d1c8a1-2c16-490e-9922-7f9c55d7d920
X-Runtime: 0.074143
X-Xss-Protection: 1; mode=block
========================= END DumpResponse =========================


========================= BEGIN DumpRequest =========================
GET /v1/solutions/66f3b82a26c248d3934fcb11e708b25f/files/hello_world_test.py HTTP/1.1
Host: api.exercism.io
Authorization: Bearer d4a3xxxx-secret-xxxxxxx59c56
Content-Type: application/json
User-Agent: github.com/exercism/cli v3.0.8 (windows/amd64)
========================= END DumpRequest =========================


========================= BEGIN DumpResponse =========================
HTTP/1.1 200 OK
Transfer-Encoding: chunked
Cache-Control: max-age=0, private, must-revalidate
Connection: keep-alive
Content-Type: text/plain; charset=utf-8
Date: Thu, 06 Sep 2018 05:42:15 GMT
Etag: W/"da0c75d4be9181852a9813271c71a17f"
Referrer-Policy: strict-origin-when-cross-origin
Server: nginx/1.10.3 (Ubuntu)
Set-Cookie: site_context=normal; domain=.exercism.io; path=/
Set-Cookie: _exercism_session=1J7CtsAjgMv%2F1BtuLUWbDtt3LO56Pz5XbAVRTSIJG4Xqlww%2F%2BweLyrEUY6VdHKT%2FWqzU%2BUaVUvGF5alGaC5raOdoVY8xW9%2BWlk4rI9pj%2FlEZRC%2FfIbx4649uJ9HtY3aqxmc3RLsCpKHZ75PZ4OK4NWBOqcZgRueldWovcBHQf8F4jwSKzDIjzB7DPPM2wStCjjHnMXkXQ0%2BxzN50MAFU4C19%2Bzcg%2BTciYNKfyfv4eGAcND4%2BSIl5xe0HIGq3yUsEbvid4CAAzJiguYRKcyo%3D--9%2FXGSokZc8sfAlDl--SsXvDulOcD3DRPVlS1dTSw%3D%3D; domain=.exercism.io; path=/; HttpOnly
X-Content-Type-Options: nosniff
X-Download-Options: noopen
X-Frame-Options: SAMEORIGIN
X-Permitted-Cross-Domain-Policies: none
X-Request-Id: 90199569-573d-4609-adad-6e392f634bee
X-Runtime: 0.068671
X-Xss-Protection: 1; mode=block
========================= END DumpResponse =========================
Downloaded to
C:\Users\Ben\Exercism\python\hello-world

This created hello-world folder holding four files.
The top three were downloaded and the fourth is metadata.

• hellow_world.py
• hello_world_test.py
• README.md
• .solution.json
  { "track":"python",
  "exercise":"hello-world",
  "id":"66f3b82a26c248d3934fcb11e708b25f",
  "url":"https://exercism.io/my/solutions/66f3b82a26c248d3934fcb11e708b25f",
  "handle":"bencoman",
  "is_requester":true,
  "auto_approve":true
  }

Now Submitting...

After making edits, uploading two files since one is not informative enough.

C:>exercism --verbose submit hello_world.py hello_world_test.py

========================= BEGIN DumpRequest =========================
PATCH /v1/solutions/66f3b82a26c248d3934fcb11e708b25f HTTP/1.1
Host: api.exercism.io
Authorization: Bearer d4a3-secret-9c56
Content-Type: multipart/form-data; boundary=c0b6fc45e5447d6cef7e2a87f4fda74d59cadd4470013168da100f1c31e5
User-Agent: github.com/exercism/cli v3.0.8 (windows/amd64)

--c0b6fc45e5447d6cef7e2a87f4fda74d59cadd4470013168da100f1c31e5
Content-Disposition: form-data; name="files[]"; filename="hello_world.py"
Content-Type: application/octet-stream

def hello():
    return 'Hello, World!'

--c0b6fc45e5447d6cef7e2a87f4fda74d59cadd4470013168da100f1c31e5
Content-Disposition: form-data; name="files[]"; filename="hello_world_test.py"
Content-Type: application/octet-stream

import unittest

import hello_world


# Tests adapted from `problem-specifications//canonical-data.json` @ v1.1.0

class HelloWorldTest(unittest.TestCase):
    def test_hello(self):
        self.assertEqual(hello_world.hello(), 'Hello, World!')

if __name__ == '__main__':
    unittest.main()

# Edited just to dirty file
--c0b6fc45e5447d6cef7e2a87f4fda74d59cadd4470013168da100f1c31e5--
========================= END DumpRequest =========================


========================= BEGIN DumpResponse =========================
HTTP/1.1 201 Created
Transfer-Encoding: chunked
Cache-Control: max-age=0, private, must-revalidate
Connection: keep-alive
Content-Type: application/json; charset=utf-8
Date: Thu, 06 Sep 2018 05:50:09 GMT
Etag: W/"44136fa355b3678a1146ad16f7e8649e"
Referrer-Policy: strict-origin-when-cross-origin
Server: nginx/1.10.3 (Ubuntu)
Set-Cookie: _exercism_session=12QdD0ohkwDsY%2FR%2BfWvuPPS8QeKIKwIJEBAfFVh28RzSnvx%2Fmra0TobHaNGzmCbn%2By0swKRh2t1sOAeOiQkNVftOSaXtr5NYr7dcrRsd6ciga%2Fz2%2FfqoIHMqMNPOklijQkW72LFuYlLHI6Nk9I%2Fghdm1Ng%3D%3D--td6UIK5czblg4N4w--6O%2F%2FoOZpbRTa%2B8rCYzCGLQ%3D%3D; domain=.exercism.io; path=/; HttpOnly
X-Content-Type-Options: nosniff
X-Download-Options: noopen
X-Frame-Options: SAMEORIGIN
X-Permitted-Cross-Domain-Policies: none
X-Request-Id: 70efb155-773f-4eb0-8a8c-66223eef0d5d
X-Runtime: 0.184868
X-Xss-Protection: 1; mode=block
========================= END DumpResponse =========================
    Your solution has been submitted successfully.
    You can complete the exercise and unlock the next core exercise at:
    https://exercism.io/my/solutions/66f3b82a26c248d3934fcb11e708b25f

[bencoman]

Haven't bumped into the HTTP PATCH verb before. Some info...
WIkipedia - Patch verb
Please. Don't Patch Like An Idiot.
RFC 5789 - PATCH Method for HTTP

"It is important to realize that the request entity to PATCH is of a different content-type than the entity that it is modifying." [ref]

[bencoman]

Just noting, running the identical command without any edits returns a bad status. The full diff shown here, and just the important part below...

========================= BEGIN DumpResponse =========================
HTTP/1.1 400 Bad Request

Making an edit to just one of the files submitted then gives good status...

========================= BEGIN DumpResponse =========================
HTTP/1.1 201 Created

[samWson]

I've been hacking away at a solution in the playground to get a rough idea of what works. I'm close but not quite there yet. This is all new territory for me. Here is my hacked together submitting client:

fileName := 'path to a Tonel file'.

client := ZnClient new
	http;
	host: 'api.exercism.io';
	method: 'PATCH';
	headerAt: 'Authorization' put: 'Bearer ', apiToken;
	contentType: ZnMimeType applicationOctetStream;
	contentType: ZnMimeType multiPartFormData;
	headerAt: 'Content-Disposition' put: (ZnMimeType main: 'form-data' sub: '' parameters: { 'name' -> 'files[]'. 'filename' -> fileName } asDictionary).

Not quite working yet as I get the error: There is no request entity yet, cannot set its content-type. I know roughly what I'm missing, I just haven't been able to find good documentation on how do to it among all the Zinc docs. I've been mainly referencing Enterprise Pharo.

[samWson]

Update on the above. I've got one step further and the client is now looking like this:

client := ZnClient new
	entity: (ZnMultiPartFormDataEntity new addPart: 'boundary=f7b01eaa5fcf167c2094054c167fd0685b596a361e868587b0095593730d');
	http;
	host: 'api.exercism.io';
	method: 'PATCH';
	headerAt: 'Authorization' put: 'Bearer ', apiToken;
	contentType: ZnMimeType applicationOctetStream;
	contentType: ZnMimeType multiPartFormData;
	headerAt: 'Content-Disposition' put: (ZnMimeType main: 'form-data' sub: '' parameters: { 'name' -> 'files[]'. 'filename' -> fileName } asDictionary);
	patch.

Not working yet. I'm only just discovering what multipart form data entities are.

[samWson]

Please excuse the rambling, but I've had an exciting crash course on multipart form data entities and I'm finally figuring out parts on the Zinc library. Our client looks like this:

apiToken := (STON fromString: 'path to user.json file' asFileReference contents) at: #token.
fileName := 'path to a exercise file of Tonel format'.

multiPartFormDataEntity := ZnMultiPartFormDataEntity new 
	addPart: (ZnMimePart fieldName: 'files[]' fileNamed: fileName).

client := ZnClient new
	entity: multiPartFormDataEntity;
	http;
	host: 'api.exercism.io';
	method: 'PATCH';
	headerAt: 'Authorization' put: 'Bearer ', apiToken.

client patch;
        response.

Helpfully the Zinc library handles all the finer details of boundaries, content types and dispositions etc. automatically. The response on this one is 404 so it's not quite right yet. I'll keep looking into it an hopefully have something more tomorrow.

[bencoman]

@samWson, looks like good progress. From my experience reported exercism/cli#732 I believe that service doesn't work with HTTP. Try HTTPS,

[samWson]

I've changed the scheme to HTTPS, but it is still a 404 response so something else is bad. Here is the transcript output:

2018-09-09 18:02:42 005 Connection Established api.exercism.io:443 46.51.205.192 831ms 
2018-09-09 18:02:42 006 Request Written a ZnRequest(PATCH /) 0ms
2018-09-09 18:02:42 007 Response Read a ZnResponse(404 Not Found text/html;charset=UTF-8 1320B) 281ms
2018-09-09 18:02:42 008 PATCH / 404 1320B 281ms

I've tweaked around with the Authorization, using the wrong token, and hard coding the correct token, just to be sure that wasn't the problem. I've made sure that the URI is using HTTPS. I'm not sure what else could be wrong here.

[bencoman]

I've had a pretty good go at this and am also stuck. I've now got the Exercism website up and running locally per https://github.com/exercism/website to facilitate observing the CLI's wire transfer in operation. I've successfully signed up to it with my github account, but it seems there is no track data loaded in the database. http://lvh.me:3000/my/tracks tells me I have 0 tracks and searching for python returns empty response. I've inquired how to load track data into into the database.

I presume I should configure the CLI like this...
C:>exercism configure --token 958b-new-secret-from-local-server-95df --api http://lvh.me:3000/v1
but I'm getting...

Error: The base API URL 'http://lvh.me:3000/v1' cannot be reached.
API returned 404 Not Found

Aside: I actually found it easier to not use docker which I struggled with, since I'm not familiar with it. Ended up running it under Windows 10 Subsystem for Linux Ubuntu 16.04 and accessing it from Chrome on Windows side.

[bencoman]

kytrinyx> We are doing a multipart fileupload. Here's a test request:

POST / HTTP/1.1
Host: 127.0.0.1:49821
Accept-Encoding: gzip
Content-Length: 459
Content-Type: multipart/form-data; boundary=2982d7b5665d520d5dfd2f7521bfbf7ef6ebf698acd9fef92130f34c5161
User-Agent: Go-http-client/1.1

--2982d7b5665d520d5dfd2f7521bfbf7ef6ebf698acd9fef92130f34c5161
Content-Disposition: form-data; name="files[]"; filename="/file-1.txt"
Content-Type: application/octet-stream

This is file 1.
--2982d7b5665d520d5dfd2f7521bfbf7ef6ebf698acd9fef92130f34c5161
Content-Disposition: form-data; name="files[]"; filename="/subdir/file-2.txt"
Content-Type: application/octet-stream

This is file 2.
--2982d7b5665d520d5dfd2f7521bfbf7ef6ebf698acd9fef92130f34c5161--

@kytrinyx, thx for the sample request above (I copied here from the other closed thread). Can you confirm the use of POST and also the root "/" with no path. Several posts above (just below the "Now Submitting..." heading ) you can see that the CLI exercism --verbose reported it was using PATCH rather than POST.

[bencoman]

@samWson, I picked up that you're also a Ruby guy. This looks related to loading data into the local web site, just don't know how to invoke it -- any clue...
https://github.com/exercism/website/blob/master/app/services/git/syncs_tracks.rb

[bencoman]

I was having trouble redirecting the CLI to my local Exercism website, then found the correct form was...
C:> exercism configure --token 958b-secret-from-local-site-95df --api http://lvh.me:3000/api/v1

[bencoman]

One more hurdle I had to get over, WireShark doesn't work on loopback, so I had to get my local server and the client running on different machines.

Now Yay! I managed to capture HTTP off the wire for a submit (regardless of empty database)

PATCH /api/v1/solutions/7818c4b34026492dba638633790f2113 HTTP/1.1
Host: lvh.me:3000
User-Agent: github.com/exercism/cli v3.0.9 (linux/amd64)
Content-Length: 279
Authorization: Bearer 958ba94b-27e2-4f2d-9f0e-14f5188395df
Content-Type: multipart/form-data; boundary=d56d661390c0485c9094bb77dfca630597ec4e947172b8ac6a7b3d1971f1
Accept-Encoding: gzip

--d56d661390c0485c9094bb77dfca630597ec4e947172b8ac6a7b3d1971f1
Content-Disposition: form-data; name="files[]"; filename="hello_world.py"
Content-Type: application/octet-stream

def hello():
    pass
   EDIT3
--d56d661390c0485c9094bb77dfca630597ec4e947172b8ac6a7b3d1971f1--

Note the extra 'api' string before the '/v1' shown in the wire capture
compared to the previous exercism --verbose captures logged above.

Now I discover that...
exercism.exe configure --token d4a3-mysecret-9c56 --api https://exercism.io/v1
doesn't work, while this does...
exercism.exe configure --token d4a3-mysecret-9c56 --api https://exercism.io/api/v1

But within Pharo, if I do...
ApiPath := 'api/v1/solutions/latest' then ExercismDownload exercise: 'hello-world'
I get the following response string...

We launched a brand new version of Exercism.\n\n
Upgrade your client with:\n\n
    exercism upgrade\n\n
Or delete the old CLI and follow the instructions\non the new site to get the new one.\n\n"}

I then reconfirmed that the following worked...
ApiPath := 'v1/solutions/latest' then ExercismDownload exercise: 'hello-world'

@kytrinyx, @iHiD, I'm confused. Maybe https://github.com/exercism/website which I installed locally is the old site?

[bencoman]

Here is where I'm up to. The request capture from the wire for following...

downloaded := ExercismDownload track: 'python' exercise: 'hello-world'.
downloaded solution inspect.     "btw, Icon changes for exercise on Exercism site"

apiToken := 'd4a37d75-mysecret-4a4770459c56'.

myEntity1 :=ZnByteArrayEntity bytes: 'This is file 1.'.
myEntity2 :=ZnByteArrayEntity bytes: 'This is file 2.'.
myPart1 := ZnMimePart fieldName: 'files[]' fileName:'/file-1.txt' entity: myEntity1.
myPart2 := ZnMimePart fieldName: 'files[]' fileName:'/file-2.txt' entity: myEntity2.
multiPartFormDataEntity := ZnMultiPartFormDataEntity new addPart: myPart1;  addPart: myPart2.

client := ZnClient new.
client
	https;
	host: 'api.exercism.io';
	headerAt: 'Authorization' put: 'Bearer ' , apiToken;
	setAcceptEncodingGzip;
	entity: multiPartFormDataEntity;
	url: 'http://api.exercism.io/v1/solutions/', (downloaded solution at: 'id').
client request headers removeKey: 'Accept'.
client patch.
client response inspect

shown below is very close to @kytrinyx sample a few posts above,

PATCH /v1/solutions/af454692256b4ce1aef52aba29a27b30 HTTP/1.1
Host: api.exercism.io
Content-Type: multipart/form-data;boundary=Boundary-Zn-QZXSRSCY
Content-Length: 372
Accept-Encoding: gzip
User-Agent: Zinc HTTP Components 1.0 (Pharo/6.0)
Authorization: Bearer d4a37d75-2da2-4300-9ec1-4a4770459c56

--Boundary-Zn-QZXSRSCY
Content-Type: application/octet-stream
Content-Length: 15
Content-Disposition: form-data;name="files[]";filename="/file-1.txt"

This is file 1.
--Boundary-Zn-QZXSRSCY
Content-Type: application/octet-stream
Content-Length: 15
Content-Disposition: form-data;name="files[]";filename="/file-2.txt"

This is file 2.
--Boundary-Zn-QZXSRSCY--

but gets a response...

500 Internal Server Error

<!DOCTYPE html>
<html>
<head>
  <title>The page you ...out that.</p>
  <p>Continue by heading back to <a href=""/"">the homepage</a>.</p>
</body>
</html>

Differences to CLI client appear to be:

• format of boundary string. But ours still seems to conform
• no space after semi-colons - ?
• different header order - but I can't believe that has an impact
• Content-Length for each part - ?

Difference to @kytrinyx sample a few posts above:

• PATCH instead of POST - Doing the same request as a POST returns a 200 Success, BUT no icon state change is visible on the web site.
• Authorization Bearer - but without it returns Unauthorized Request errors

[kytrinyx]

Can you confirm the use of POST and also the root "/" with no path.

Shoot, no, that was from when I was working out the original request and had made a stand-alone app that only had one endpoint that accepted multi-part file uploads. So sorry about that rabbit hole!

[kytrinyx]

I presume I should configure the CLI like this...

C:>exercism configure --token 958b-new-secret-from-local-server-95df --api http://lvh.me:3000/v1

but I'm getting...

Error: The base API URL 'http://lvh.me:3000/v1' cannot be reached.
API returned 404 Not Found

The --api value needs to be a URL you can hit in your browser. Are you able to go to http://lvh.me:3000?

[kytrinyx]

You're right that this uses patch not post (the post goes back to my original stand-alone test).

[kytrinyx]

Note the extra 'api' string before the '/v1' shown in the wire capture compared to the previous exercism --verbose captures logged above.

Ah, yes. We need to add documentation about this.

The production API is delivered from api.exercism.io/v1, but locally we don't mount the API to a different place, so it needs to be accessed from /api/v1

[kytrinyx]

I'm confused. Maybe https://github.com/exercism/website which I installed locally is the old site?

No, this is the new site. The error message is the one that you get if you hit an endpoint that is not valid for the API. So for example: http://api.exercism.io/api/v1/solutions/latest (notice that it has both api.exercism.io and the /api/ path segment). If you hit a valid URL: https://api.exercism.io/v1/solutions/latest then it will return a valid JSON response (in this case an empty JSON object, which is likely because we're hitting the endpoint without being authenticated. Presumably we'd actually want an error here).

[kytrinyx]

Authorization Bearer - but without it returns Unauthorized Request errors

When I was originally developing the client I tested with both Authorization Bearer and Authorization Token and both worked.

[kytrinyx]

I'm really not sure what is causing the 500 error when you hit production as described in #96 (comment). I can't find that error in our logs.

[samWson]

@samWson, I picked up that you're also a Ruby guy. This looks related to loading data into the local web site, just don't know how to invoke it -- any clue...
https://github.com/exercism/website/blob/master/app/services/git/syncs_tracks.rb

@bencoman at a guess it looks like it is for keeping the language tracks synchronized on the website (you wouldn't need to auto approve hello world on the users computer right?). I get lost where the code reaches out of the class with Git::ProblemSpecifications.head.fetch! on line 26 and Git::SyncsTrack.sync!(track) on line 40.

For invoking it do you mean at the Rails console using the class itself? The class method self.sync is what your after. Almost everything else is private. You could use it at the console like so: Git::Syncstracks.sync(<collection of tracks>). There is an example of this here on line 28. This looks like it is the only place it is used in the code (other than the tests).

[iHiD]

You probably want:

bundle exec rake git:fetch
bundle exec rake git:sync

[bencoman]

Further discussion on getting a local copy of the Exercism website running moved to #103.

With that, I isolated our "500 INTERNAL SERVER ERROR" to a missing "space" in the "Content-Disposition:" of the Mime Part (as observed in raw HTTP Request captured by Wireshark.)
i.e. broken...
Content-Disposition: form-data;name="files[]";filename="myfile.txt"
working...
Content-Disposition: form-data; name="files[]"; filename="myfile.txt"

A diff of the full requests plus library code change is here... https://www.diffchecker.com/ETIfCUzx

[bencoman]

PHARO SUBMIT TESTING RESULTS (log success directly in this post rather than spread across posts)
2018.09.14.BenComan Windows 10 works
2018.09.14.BenComan Linux Mint 7.3 64-bit works
2018.09.14.TimM OSX 10.13.5 (High Sierra) works

[macta]

Nice one @bencoman - but the question is, is this a bug in that Pharo method for ZnMimePart - or is this a parsing bug in the exercism server? The mozilla docs for content-disposition don't mention a space is required, just that all parameters are separated by a ';'. (although all their examples show a space after each parameter)

We can of course get Pharo fixed (or add our own extension method to workaround this) - but it seems strange that space is required. I guess we should ask sven.

[samWson]

@bencoman cool. I've gone through your steps above and now I finally understand. I see the issue that we don't know where the bug is, Pharo or Exercism. I suggest in the short term we do a one time fix to our own code to get around this. That way we can keep moving forward and it's not hard to reverse if we need to.

[macta]

I think we can add our own extension method (with an ex prefix) as I think we call it vs the library right?

I have a long flight coming up in a few days, and can look at replacing the shell out code.

Looks pretty easy given Ben’s example.

Or is one of you guys itching to do it?

[bencoman]

Agree on short term action - we should adopt that method by changing its method-protocol to "*ExercismTools".

Now with the ancillary difficulties failing to get the Dev Local Website going under WSL, and doing it again on an old Linux box and then experimenting/analysing the Exercism CLI HTTP traffic, I burnt four FT days on it when I should have been working on something else. So if someone could do the work to implement my POC that would be great.

The final piece of the puzzle of how/where to store the solution metadata between the Download and the Submit has one option implemented in PR #104.

[bencoman]

Cool, thanks Sam.

push up some variables from the the other DownloadCommand that Ben refactored this too.

To provide that info, I've merged PR #104 into master. As I commented in that PR, there may be a better way, but this is sufficient for now to provide the solutionId that Sam needs.

After pulling down the latest master the Playground script can be rewritten...

"DOWNLOAD EXERCISE"
ExercismDownload exercise: 'hello-world'.

"SUBMIT EXERCISE"
solutionId := (ExercismSubmit solutionDataForExercise: 'hello-world') at: 'id'.
iteration := 1 + (iteration ifNil: [ 0 ]).   "no changed entities ==> Response(400 Bad Request)"
solutionEntity1 :=ZnByteArrayEntity bytes: 'MY SOLUTION #', iteration printString.
solutionEntity2 :=ZnByteArrayEntity bytes: 'SECOND PART OF MY SOLUTION.'.
solutionPart1 := ZnMimePart exercismFieldName: 'files[]' fileName: 'mysolution.py' entity: solutionEntity1.
solutionPart2 := ZnMimePart exercismFieldName: 'files[]' fileName: 'support.py' entity: solutionEntity2.
multiPartFormDataEntity := ZnMultiPartFormDataEntity new.
multiPartFormDataEntity 
    addPart: solutionPart1 ;
    addPart: solutionPart2. 
client := ZnClient new.
client
	headerAt: 'Authorization' put: 'Bearer ' , apiToken ;
	url:  apiUrl / solutionId ;
	entity: multiPartFormDataEntity.
client patch.
client response inspect. "also check  website is updated"

Now there are a few cases that may be useful:
a. Since our classes==files, specify the classes to Submit. e.g. ExercismSubmit classes: {HelloWorld}.
b. Since ultimately may invoke this by right-clicking on the exercise-package-tag in System Browser, specify the package-tag to submit. ExercismSubmit package: 'Exercism-HelloWorld'
c. Submit whole exercise via Exercism exercise-name e.g. ExercismSubmit exercise: 'hello-world'

[bencoman]

After that merge, you can now do...
solutionId := (ExercismSubmit solutionDataForClass: HelloWorld) at: 'id'.

[samWson]

@bencoman @macta related to using the UI to submit exercises, the message ExTonelWriter>>#sourceDir: gets sent in ExercismManager>>#submitToExercism:, but this method is not implemented anywhere in ExTonelWriter hierarchy. I tried removing it to see what would happen and it still seems like it is required.

(writer := ExTonelWriter new)
		packageDir: (submissionDirectoryRef relativeTo: trackDirectoryRef) exPathString;
		sourceDir: trackDirectoryRef;
		writeSnapshot: packageOrTag snapshot.

I'm wondering at this point if it is a method we made ourselves and have accidentally deleted it, or my image is missing some code from a branch or just somehow is not up to date.

The same code is also used in ExercismGenerator>>#generateSourceFilesFor:to:.

[samWson]

I've been looking around over the last week on this and I had a thought. We had to make some changes to TonelWriter to make it work with an in memory file system for PR #110. I didn't include the changes to TonelWriter in my last commit for that PR. They are summarised here.

@bencoman since it looks like your going to be busy for the time being, if you don't mind I'll make a PR for the changes to TonelWriter based on your comment linked above.

After that I'll keep looking around to try and make some more progress.

[samWson]

It turns out the fix for TonelWriter was simple (I hope this doesn't come back to bite me).

I've pushed PR #112 which should fix submitting exercises from the GUI. If there is any OSProcess still in the repository it should be no longer in use and save to remove.

I think we finally may have a working product for all three operating systems now.

I'm going to take a break from coding for the next few days. We have a lot of issues open and I'd like to go back and review them so we can keep this project pointed in the right direction.

[samWson]

#112 is merged. I think this issue is finally fixed. We should have a working submit solution for all operating systems now.

[bencoman]

btw, tracked the issue back to Rack, a Ruby library used by Sinatra, that is used by Exercism.
Opened rack/rack#1386