unfoldingword / gateway-translate

A tool for editing and translating text in USFM markup

gatewaytranslate

This application is for use by Gateway Language translators.
Explore the docs »

Go to Application · Report Bug · Request Feature

Table of Contents

1. About The Project
   • Built With
2. Getting Started
   • Prerequisites
   • Installation
3. Usage
4. Roadmap
5. Contributing
6. License
7. Contact
8. Acknowledgments

About The Project

Login Page

Setup Account

View Texts

Purpose To provide a tool for Translation Teams to edit and translate Scripture Texts, which are typically in USFM format.

Problem There are few (if any) web based editors for USFM markup, especially focused on translation activities.

Scope

• Current scope is focused on editing USFM text and doing alignments with the original language texts.
• It is envisioned that this tool also:
  • Enable translation of Bibilical Resources (translation notes, questions, etc.)
  • Supersede the existing tooling in unfoldingWord® for translation tasks

Background This project uses components from the Open Components Ecosystem (OCE) extensively. In particular it relies on Proskomma, a scripture runtime engine for the editor component itself. The latter is also contributed to the OCE community and we welcome others to reuse it.

(back to top)

Built With

• Next.js
• React.js

(back to top)

Getting Started

Installation/First Steps

1. Get a DCS account
2. Clone the repo

   git clone https://github.com/unfoldingword/gateway-translate.git

3. Install NPM packages

   yarn install

4. Start the local server

   yarn dev

5. Visit localhost:3000

(back to top)

Roadmap

See the open issues for a full list of proposed features (and known issues).

This project uses Zenhub to manage the roadmap.

(back to top)

Contributing

Contributions are what make the open source community such an amazing place to learn, inspire, and create. Any contributions you make are greatly appreciated. Guidelines for external contributions.

You can also simply open an issue with the tag "enhancement". Don't forget to give the project a star! Thanks again!

If you would like to fork the repo and create a pull request.

1. Fork the Project
2. Create your Feature Branch (git checkout -b feature/AmazingFeature)
3. Commit your Changes (git commit -m 'Add some AmazingFeature')
4. Push to the Branch (git push origin feature/AmazingFeature)
5. Open a Pull Request

Finally, you can point the application to either the QA or Production servers this way:

To use QA add the "server=qa" parameter to the URL: https://gateway-edit.netlify.app/?server=qa.

This is a toggle and will stay pointed at QA until another setting is applied.

To point to production: https://gateway-edit.netlify.app/?server=prod

(back to top)

License

Distributed under the MIT License. See LICENSE for more information.

(back to top)

Contact

Cecil New

• Twitter @mandolyte
• Discord Cecil#5551

Project Link: https://github.com/unfoldingword/gateway-translate

(back to top)

gateway-translate's Issues

Obtain usfm text for scripture card

Once a book is selected by the user, the file can be selected.

The state books must be converted from an array of strings (of bookIds) to an array of objects, where the object will have:

• bookId
• literal or simplified (LT | ST)
• the usfm text from DCS
• url (if applicable)
• other things?

Suggestion: do not let app header scroll off page

Currently, the app header scrolls off page when working with cards on second row (with default sizing).

UI Issues:Text boxes needs to be aligned

In Bug Report or Feedback menu (Dashboard)

Steps:

1. Open the app (https://gateway-translate.netlify.app/)
2. Click on the left dashboard icon.
3. Select the Bug Report or Feedback menu.
4. Text boxes need to be aligned.
   

Accounts Setting (Dashboard)

1. Open the app (https://gateway-translate.netlify.app/)
2. Click on the left dashboard icon.
3. Select the Accounts Setting menu.
4. Text boxes need to be aligned. text boxes are too close. so the text is overlapping
   

DCS Branch Management

Pseudo-code:

On Read

if user branch exists then attempt retrieval of USFM from the user branch;
otherwise use the master branch

On Save

if user branch exists then post/put edited USFM to the user branch;
otherwise create the branch first, then do the post/put.

Text edit UI from URL

When text is from a non-DCS source, should it be non-editable?

At present, it can be viewed and changed, but cannot be saved.

Subsequent Saves do not work

This is almost certainly due to not refreshing the data from DCS. This must be done in order to see the SHA value which is required in order to overwrite an existing file. So the flow should be:

• save to branch
• fetch from branch
• update data.content value in app context

Update README

Break editor out into RCL components

Break editor out into RCL components

Open/Save offline USFM files

Not able to close the cards

In some cases, the cards that are opened are not able to close using the "x" icon

Steps:

1. Open the app and log in
2. Add/Open more than 6 different books in the window.
3. Try to open one or two same books that are opened.
4. Close the card from the last.
5. You may not close the cards.
   Also, the layout is changed.
   

Bug: check if book exists in a user branch on opening

If the user had previously edited the file, then a subsequent open of the book should pick up where they left off - by using the copy in their user branch

Add save button

Add save button to the card and the function that should execute when clicked.

Create file scenarios

At present, when a file is fetched there is no provision to create a file from scratch.

Should there be?

In issue #21, it is proposed that gT should have a setting for the "source" file in addition to the current "target" file. Thus if Ruth is selected, then two windows would be opened:

• one with the source for Ruth, in the language they are translating from
• one with the target for Ruth, in the language they are translating to

In this scenario, what happens if the target does not exist (in either the master or their user branch)?
@birchamp

Navigation and Syncronization of books

History - This will allow the user to see a list of the locations of that they have been to recently.
Sync the cards that contain the same Bible book.
Allow user to open other books.

Editing aligned USFM Scripture

Open/Close
Save
Insert/edit verse numbers
Insert/remove headings
Edit text
USFM validation/strictness

Add org and repo to "add book" dialog

The organization should default to the account settings organization. If repo is supplied, then the input should be a filepath. Is the filepath something you type? or should it be a file selector?

Add option to supply a URL to open the USFM text

Close cards

Allow users to close open cards. We already done done it.

Editor should offer font and size selection

This will be the most useful for complex scripts. This should look and feel similar to how gEdit handles card features with a popup. Users will need to be notified that some complex fonts will not display correctly in browsers other than Firefox.

Cannot load two books by URL in same session

This error appears on second book:

TypeError: Cannot read properties of undefined (reading 'outerHTML')
    at t.embedPreviewTextInGrafts (_app-7d92365e7b11cd77.js:204:202584)
    at _app-7d92365e7b11cd77.js:204:188935
    at Object.useMemo (framework-0ba0ddd33199226d.js:1:69563)
    at n.useMemo (framework-0ba0ddd33199226d.js:1:136651)

Search and Replace RCL

Add scripture picker

Plan:

• Make the "home" page of the app to be a book picker
• Make it an auto-complete drop-down list of the 66 books of the bible
• Make a radio button to select either LT or ST

Once selected, then:

• attempt to retrieve (using server, org, lang, LT|ST, book-usfm
• on success, store in the admin context and route to editing page (a dummy for now) to show it in a text box
• on failure, show status and allow user to select another book if they wish.

Login dependency on gitea-react-toolkit

Replace use of gitea-react-toolkit with dcs-js; then remove gitea-react-toolkit dependency from package.json

Clear Custom URL field after it is added

The URL stays in the field after the book is added and user needs to clear it manually to add another URL.

Also, a X button can be added to remove the URL, if the user wants to remove the copied URL.

Update navigation based on current position in the text

When a user moves a vertical scroll in the text of a chapter other than the one it is currently in, the navigation bar should update and the parallel scripture cards should move to display that area of the text as well.

DoD:

Clicking inside a particular verse within the text will update the navigation bar and other parallel scripture cards.

Alignment Component

I'll post my thoughts on how this might work in a subsequent comment.

Breakout notification of broken alignments

Broken alignment notification UI needs to be separated from search and replace. This is a tech debt issue.

SPIKE: Create a Progressive Web App

Are there changes needed to our stack?
Do we need to pick one technology over another? If so, what are the pros/cons

Add Book, chapter, verse navigation

Users need to be able to navigate to books that have already been added manually, chapters and verses . See the epic for the mockup #16

All cards of the same book get updated. For now we will not implement the link button.

DoD

users can navigate to Book, chapter, and verse using a top navigation bar

Hide menu in dashboard?

Show the menu/controls(account settings, reset resource layout, bug or report extra..) that are in the dashboard only when the user logs in.?

Handle 404 error on non-existent file access

For example, using:

• org es-419_gl
• lang: es-419
• try to add, say the book of Judges.

Make card/bookId unique for URL fetches

At present, the ID seems to be upper case bookId followed by the word custom. But:

• how is the bookId determined? inspection of URL is not likely to be reliable
• the word custom works for one such fetch, but what about two of them?

Consider removing the line that says "Main Sequence Editor"

Book template generation

Generate the empty USFM for a new book according to a chosen versification.
\id XXX
\p
\c 1
\v 1

Idea: Add a download feature

If text obtained by URL is edited, the user may not be able to save the changes. A download feature will enable the user to save the results locally and then use the downloaded results in some external workflow.

Save Function

Save Function Design

Scope

This design is intended to cover only the function that stores data back to DCS in a user branch. It does not cover any UI/X look and feel. It will impliment using the dcs-js package.

Context

During the edit of a Scripture text, the user may wish to save current work at any time. The app will respond to a save request by the user by doing the following:

• Inputs:
  • Type of content: ['usfm' , 'perf']
  • Content to be saved
  • BookId to be saved
  • DCS user name
• Actions:
  • If save, convert PERF to USFM (unless Epitelete does this)
  • Test for existence of user branch, creating if it doesn't exist
  • Save content to user branch
• Outputs:
  • Console log showing save details
  • Boolean return indicating success (or failure)
  • Message with failure details (to be used in an error popup)

USFM by URL

USFM by URL

Goal: a translator will be able to view and/or edit USFM text obtained via URL.

There are three general use cases.

Use Case 1

The USFM text comes from DCS, but user does not have access to save the changes. For example, the Original Languages Greek and Hebrew texts, wherein the user, unless thay are a member of the unfoldingWord organization, will not be able to save any changes. This use case is expected to common for any translator who know Greek or Hebrew.

Use Case 2

The USFM text comes from DCS and the user has access to save the changes, since they are a member of the owning organization. This would be a common use case for a translator who wanted to refer to legacy texts stored on DCS. This would also be the use case for making changes to the Greek or Hebrew texts.

Use Case 3

The USFM text does not come from DCS. This use case might be of use to anyone who wished to use our tools. Since the edits could not be saved, gT will need to provide a download option for the edited text.

Note on Original Languages

Since Greek and Hebrew is anticipated to be common use case, gT should conside a built in way to obtain the original language texts.

Considerations and Constraints

1. If a DCS text is from a "release" URL, then even if the user has edit rights, we should only permit edits to be done from URLs from the master branch.
2. The book ID must be a valid one. (reference)
3. For DCS texts, inspection of the URL will yield both an organization and a repo. These might be used to formulate the required doc set ID.
4. For non-DCS texts, creation of the required doc set ID will take some thought.
5. Since we cannot always determine the book ID from the URL, consider forcing the user to specify the book from the drop down list. If we wish to support apocryphal or other non-canonical texts, our drop down should contain all the books from the reference in item 2 above.
6. By requiring the user to supply the book ID, then these texts will be able to participate in the linked card navigation and synchronization. (see issue 16)

Translation Source Organization

Currently, the Account Settings page does not prompt for a source for translations.

Concept:

• Add (and require) a prompt for the org from which the translation work will be done
• When user opens, say, Titus Literal, then two windows should open:
  • the source literal text
  • the target literal text
• If the target text does not exist, then a template should be opened to give them a place to start

Notes:

1. This implies that the windows will always be paired.
2. It implies that users cannot start without the source text - it must exist. They could not, for example, attempt translation from a paper copy of the source.

@birchamp After the call today, this need occurred to me. Might be overly restrictive... thoughts?

Deploy to netlify

PDF rendering

Users should be able to get a PDF from a Book of scripture that they currently have open.

DoD:

Users can output scripture to a PDF.

Bug: non-existent book included in array

@superdav42 I thought that perhaps it was because "obs" was in the list. But I filtered it out. So the list starts at 66 elements. And at the point where I pick a repository, suddenly it has 67 elements in the array and the first one is spurious and is causing the "undefined" error to show in the console.

Here is the mods I made to the book selection list function:

export function bookSelectList(books = null) {
  if ( ! books) {
    books = Object.keys(BIBLE_AND_OBS).filter( (b) => b !== 'obs')
  }
  const _books = books.map(
    (bookId) => ({ id: bookId, name: BIBLE_AND_OBS[bookId] }),
  )
  console.log("books:",_books)
  return books.map(
    (bookId) => ({ id: bookId, name: BIBLE_AND_OBS[bookId] }),
  )
}

Mixed use of @mui and material-ui

Suggest using only @mui, which is v5 - an upgrade of material-ui which is v4 of the material components.

Perhaps also upgrade to latest react at the same time?

And clean up the package json to remove unused packages.

Work started in branch debt-cn-pkg-cleanup

Users should be notified that alignments are broken

When editing aligned text, users should be informed when alignments have been broken. The component needs to be integrated into the app.

DoD:

The app displays information about broken alignments

Add option to select either Literal or Simplified

Plan: to the add book form, add two radio buttons for the two cases. Then use than when inferring the repo to fetch from.

Bug: Cannot open a scripture text twice

To replicate:

1. login and use org unfoldingWord and lang English
2. Open a text, say Titus, literal
3. Click on the "X" to close the card
4. Re-open Titus

Now the card will appear, but the contents will be empty

Failed to load the scripture card after login-in back

1. Login to the app, open a new book and make some edits and save.
2. Logout
3. Log back in and Note that the scripture card added in step 1 shows blank. See the errors in the console:
   

Bug on local storage

Local storage was being used to persist book content; but local storage quota is too low. Plus, book content should not be persisted across sessions. This is a bug. Change to normal useState().

Also, for large books it takes time to populate the card. Show a circular progress indicator while waiting.

Default to read-only files except for user's default repo

When a card is opened, the text can be viewed read-only (R/O) or read-write (R/W).

• Text comes from user's current Org/lang - open R/W
• Text comes from local machine - open RW
• Text comes from anywhere else - open RO.

DoD:

When the file meets the above criteria it is opened as editable. Otherwise, it is opened as read-only.

Errors on the console when adding book card through URL

1. Added Book Gen-Literal. Looks fine.
2. Added Book Gen-Simplified. Looks fine.
3. Added Book Gen using this URL: https://qa.door43.org/unfoldingWord/en_ult/src/branch/master/01-GEN.usfm
   The card shows loading at first and then shows blank with errors in the console.
   

Allow "connect as guest" option on the login page

Since the tool is able to work with non-DCS data or even data that is uploaded from local storage, a login for DCS isn't needed for these use cases. This should be a link or button on the login prompt. Developers should be able to configure whether or not guest login is available.