Get error trying to `git clone` a repository about githttpserver

[petersalomonsen]

The GIT_PROJECT_ROOT is a path to a local folder on your machine ( or on the server that you deploy this web application to ) that you want to clone repositories into your browser.

For example there's a server deployed to https://wasm-git.petersalomonsen.com and there's a repository named test, so if you type https://wasm-git.petersalomonsen.com/testinto the field with the clone button you can then try cloning into your browser.

The reason you cannot clone directly from github is because of CORS restrictions on the github side. You can however get around this by having a proxy to github on your localhost. An example of such a proxy is here:

https://github.com/petersalomonsen/wasm-git/blob/master/examples/webserverwithgithubproxy.js

Let me know if this gets you further, otherwise I'm happy to provide more details.

[nikitavoloboev]

Oh wow, thank you this worked. Will dig in more now. :)

[nikitavoloboev]

Assume issue is due to my node version being different than yours? Do you know which version it should be?

Thank you.

[nikitavoloboev]

Could be also that I am on macbook with ARM CPU?

~/clones/githttpserver master
❯ node -v
v17.9.0

~/clones/githttpserver master
❯ fnm install 16
Installing Node v16.19.1 (arm64)

Trying older versions now.

[nikitavoloboev]

Running with https://bun.sh doesn't crash but prints nothing.

I assume a server should be started.

[nikitavoloboev]

Also it's not clear from readme but how should GIT_PROJECT_ROOT variable look like?

GIT_PROJECT_ROOT=/Users/nikiv/src/lib/ts work?

/Users/nikiv/src/lib/ts is a path that has a .git folder inside.

[nikitavoloboev]

ah I just realized port was active, changed to 5002 and it worked with node 17 even.

[nikitavoloboev]

Ok but now it just starts running but logs nothing.

Shouldn't this log run?

I am not sure how I can test the git. I basically want to use git-wasm for a web app that git clones a repository so I can edit some files in the repo. All done using https://github.com/petersalomonsen/wasm-git

[nikitavoloboev]

ah wait nvm I should of just went to that port.

[nikitavoloboev]

Can you help me understand, where can I see in code of githttpserver for how to git clone a specific repository with wasm-git. The UI for githttpserver is bit confusing to help understand what repo is being cloned.

[nikitavoloboev]

For example trying to git clone an existing repo (https://github.com/nikitavoloboev/ts) I get an error

[nikitavoloboev]

Thank you for offering help.

So I read through webserverwithgithubproxy.js

But I never wrote a proxy server so I am trying to understand it line by line. I created a fork with my changes, I only moved options now.

https://github.com/nikitavoloboev/githttpserver/blob/master/server/githttpserver.js#L38

If I try the example in wasm-git, I get this:

I wanted to get an understanding of what code is actually needed to create a proxy.

I originally thought I could run proxy server on separate port and githttpserver on another and it would work. But it seems I have to instead try to redo the proxy the same file as githttpserver.

[nikitavoloboev]

Or wait perhaps I need to do this. Start server webserverwithgithubproxy.js on say port 5003 because 5000 is always taken for me on mac.

This will act as proxy.

Then I need to make change to https://github.com/nikitavoloboev/githttpserver/blob/master/server/githttpserver.js to if it's a github url that you are trying clone, go through that proxy?

I am not too sure how to do that though or if that mental model is right.

Thank you for the help. ♥️

[petersalomonsen]

Hi,

If you want to make a proxy to github out of https://github.com/nikitavoloboev/githttpserver/blob/master/server/githttpserver.js you can replace the code that connects to your local file system with the proxy code to github. I've made it for you below, and I've also tested this on my own server.

To clone a repository from github, use the normal github https URL, but replace https://github.com with http://localhost:5002

import { checkPermission, PERMISSION_OWNER, PERMISSION_CONTRIBUTOR, PERMISSION_FREE } from './checkpermission.js';

import { createServer } from 'http';
import https from 'https';
import { exists, readFile, mkdir } from 'fs';
import { exec } from 'child_process';
import * as cgi from 'cgi';
import { checkIfQuotaIsExceeded } from './quota.js';

function createCGI(accountId) {
    return cgi.default('git', {
        args: ['http-backend'],
        stderr: process.stderr,
        env: {
            'GIT_PROJECT_ROOT': process.env.GIT_PROJECT_ROOT,
            'GIT_HTTP_EXPORT_ALL': '1',
            'REMOTE_USER': accountId
        }
    });
};

const publicdir = `${process.cwd()}/public`;

createServer(async (request, response) => {
    let path = request.url.substring(1).split(/\?/)[0];

    if (path === '') {
        path = 'index.html';
    }

    response.setHeader("Access-Control-Allow-Origin", "*");

    // TODO: proxy github.com to avoid CORS issue
    const options = {
        hostname: 'github.com',
        port: 443,
        path: request.url,
        method: request.method,
    };

    if (request.method === 'OPTIONS') {
        response.setHeader("Access-Control-Allow-Headers", "*");
        response.statusCode = 200;
        response.end();
    } else if (request.url.indexOf('git-upload') > -1 ||
        request.url.indexOf('git-receive') > -1) {

        const options = {
            hostname: 'github.com',
            port: 443,
            path: request.url,
            method: request.method,
        };

        console.log(`Proxying ${options.method} request to ${options.hostname} with path ${options.path}`);
        const proxy = https.request(options, function (res) {
            res.pipe(response, {
            end: true
            });
        });

        request.pipe(proxy, {
            end: true
        });
    } else if (await new Promise(resolve => exists(`${publicdir}/${path}`, res => resolve(res)))) {
        if (path.indexOf('.js') === path.length - 3) {
            response.setHeader('Content-Type', 'application/javascript');
        } else if (path.indexOf('.wasm') === path.length - 5) {
            response.setHeader('Content-Type', 'application/wasm');
        } else {
            response.setHeader('Content-Type', 'text/html');
        }
        response.end(await new Promise(resolve => readFile(`${publicdir}/${path}`, (err, data) => resolve(data))));
    } else {
        response.statusCode = 404;
        response.end("not found");
    }
}).listen(5002);