microsoft / form-recognizer-toolkit

Form Recognizer Toolkit (FRTK) provides a set of components and features to accelerate development based on Form Recognizer service.

Project

This repo has been populated by an initial template to help get you started. Please make sure to update the content to build a great experience for community-building.

As the maintainer of this project, please make a few updates:

• Improving this README.MD file to provide a great experience
• Updating SUPPORT.MD with content about this project's support experience
• Understanding the security reporting process in SECURITY.MD
• Remove this section from the README

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.

When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

Trademarks

This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft trademarks or logos is subject to and must follow Microsoft's Trademark & Brand Guidelines. Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. Any use of third-party trademarks or logos are subject to those third-party's policies.

form-recognizer-toolkit's Issues

readBinary should request an arraybuffer?

Hi there, in Client/src/providers/storageProvider, should the readBinary function request an arraybuffer as the response? I think the axios default is a json, and this causes problems when reading a pdf for example.

    public async readBinary(filename: string, ignoreNotFound?: boolean): Promise<Buffer | undefined> {
        try {
            const api = `${serverUrl}/files/${filename}`;
            const result = await getWithAutoRetry(api, {
                responseType:"arraybuffer" //Added this
            }
        );
            return result.data;
        } catch (exception) {
            this.storageErrorHandler(exception, ignoreNotFound);
        }
    }

ServiceRequestError: No connection adapters were found

Hi,

I'm trying to run the /SampleCode/Python/sample_figure_understanding.ipynb file, but when it tries to call the Azure Document AI REST API, it throws the following error.

ServiceRequestError: No connection adapters were found for '"https://xxxxxxx-westus2.cognitiveservices.azure.com/" /documentModels/prebuilt-layout:analyze?api-version=2024-02-29-preview&outputContentFormat=markdown# Your Azure Document Intelligence resource's endpoint value./documentintelligence'

I've tried deploy the Document AI service in EastUS and West US2, same issue.
I've also tried to execute sample API call as described in the REST API 2024-02-29-preview documentation, same issue.

any advise ?

thanks in advance,

regards,
Alex

Unable to start the project

2023-12-12T08_07_33_365Z-debug-0.log
Hi, I was unable to start the project. Showing lot of dependency errors

Client installation fails with Visual Studio 2022 CE and Python 3.10

When trying to run npm i for the client, the installation routine stops:

npm ERR! gyp info find VS "C:\Program Files\Microsoft Visual Studio\2022\Community"
npm ERR! gyp info find VS run with --verbose for detailed information
npm ERR! gyp info spawn C:\Program Files\Python310\python.exe
npm ERR! gyp info spawn args [
npm ERR! gyp info spawn args 'C:\Users\USERNAME\AppData\Roaming\nvm\v18.12.1\node_modules\npm\node_modules\node-gyp\gyp\gyp_main.py',
npm ERR! gyp info spawn args 'binding.gyp',
npm ERR! gyp info spawn args '-f',
npm ERR! gyp info spawn args 'msvs',
npm ERR! gyp info spawn args '-I',
npm ERR! gyp info spawn args 'C:\Users\USERNAME\Programming\FRT_Labelling\Form-Recognizer-Toolkit\SampleCode\LabelingUX\Client\node_modules\canvas\build\config.gypi',
npm ERR! gyp info spawn args '-I',
npm ERR! gyp info spawn args 'C:\Users\USERNAME\AppData\Roaming\nvm\v18.12.1\node_modules\npm\node_modules\node-gyp\addon.gypi',
npm ERR! gyp info spawn args '-I',
npm ERR! gyp info spawn args 'C:\Users\USERNAME\AppData\Local\node-gyp\Cache\18.12.1\include\node\common.gypi',
npm ERR! gyp info spawn args '-Dlibrary=shared_library',
npm ERR! gyp info spawn args '-Dvisibility=default',
npm ERR! gyp info spawn args '-Dnode_root_dir=C:\Users\USERNAME\AppData\Local\node-gyp\Cache\18.12.1',
npm ERR! gyp info spawn args '-Dnode_gyp_dir=C:\Users\USERNAME\AppData\Roaming\nvm\v18.12.1\node_modules\npm\node_modules\node-gyp',
npm ERR! gyp info spawn args '-Dnode_lib_file=C:\\Users\\USERNAME\\AppData\\Local\\node-gyp\\Cache\\18.12.1\\<(target_arch)\\node.lib',
npm ERR! gyp info spawn args '-Dmodule_root_dir=C:\Users\USERNAME\Programming\FRT_Labelling\Form-Recognizer-Toolkit\SampleCode\LabelingUX\Client\node_modules\canvas',
npm ERR! gyp info spawn args '-Dnode_engine=v8',
npm ERR! gyp info spawn args '--depth=.',
npm ERR! gyp info spawn args '--no-parallel',
npm ERR! gyp info spawn args '--generator-output',
npm ERR! gyp info spawn args 'C:\Users\USERNAME\Programming\FRT_Labelling\Form-Recognizer-Toolkit\SampleCode\LabelingUX\Client\node_modules\canvas\build',
npm ERR! gyp info spawn args '-Goutput_dir=.'
npm ERR! gyp info spawn args ]
npm ERR! gyp info ok
npm ERR! gyp info it worked if it ends with ok
npm ERR! gyp info using [email protected]
npm ERR! gyp info using [email protected] | win32 | x64
npm ERR! gyp info spawn C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Current\Bin\MSBuild.exe
npm ERR! gyp info spawn args [
npm ERR! gyp info spawn args 'build/binding.sln',
npm ERR! gyp info spawn args '/clp:Verbosity=minimal',
npm ERR! gyp info spawn args '/nologo',
npm ERR! gyp info spawn args '/p:Configuration=Release;Platform=x64'
npm ERR! gyp info spawn args ]
npm ERR! gyp ERR! build error
npm ERR! gyp ERR! stack Error: C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Current\Bin\MSBuild.exe failed with exit code: 1
npm ERR! gyp ERR! stack at ChildProcess.onExit (C:\Users\USERNAME\AppData\Roaming\nvm\v18.12.1\node_modules\npm\node_modules\node-gyp\lib\build.js:203:23)
npm ERR! gyp ERR! stack at ChildProcess.emit (node:events:513:28)
npm ERR! gyp ERR! stack at ChildProcess._handle.onexit (node:internal/child_process:291:12)
npm ERR! gyp ERR! System Windows_NT 10.0.19044
npm ERR! gyp ERR! command "C:\Program Files\nodejs\node.exe" "C:\Users\USERNAME\AppData\Roaming\nvm\v18.12.1\node_modules\npm\node_modules\node-gyp\bin\node-gyp.js" "build" "--fallback-to-build" "--update-binary" "--module=C:\Users\USERNAME\Programming\FRT_Labelling\Form-Recognizer-Toolkit\SampleCode\LabelingUX\Client\node_modules\canvas\build\Release\canvas.node" "--module_name=canvas" "--module_path=C:\Users\USERNAME\Programming\FRT_Labelling\Form-Recognizer-Toolkit\SampleCode\LabelingUX\Client\node_modules\canvas\build\Release" "--napi_version=8" "--node_abi_napi=napi" "--napi_build_version=0" "--node_napi_label=node-v108"
npm ERR! gyp ERR! cwd C:\Users\USERNAME\Programming\FRT_Labelling\Form-Recognizer-Toolkit\SampleCode\LabelingUX\Client\node_modules\canvas
npm ERR! gyp ERR! node -v v18.12.1
npm ERR! gyp ERR! node-gyp -v v9.3.0
npm ERR! gyp ERR! not ok
npm ERR! node-pre-gyp ERR! build error
npm ERR! node-pre-gyp ERR! stack Error: Failed to execute 'C:\Program Files\nodejs\node.exe C:\Users\USERNAME\AppData\Roaming\nvm\v18.12.1\node_modules\npm\node_modules\node-gyp\bin\node-gyp.js build --fallback-to-build --update-binary --module=C:\Users\USERNAME\Programming\FRT_Labelling\Form-Recognizer-Toolkit\SampleCode\LabelingUX\Client\node_modules\canvas\build\Release\canvas.node --module_name=canvas --module_path=C:\Users\USERNAME\Programming\FRT_Labelling\Form-Recognizer-Toolkit\SampleCode\LabelingUX\Client\node_modules\canvas\build\Release --napi_version=8 --node_abi_napi=napi --napi_build_version=0 --node_napi_label=node-v108' (1)
npm ERR! node-pre-gyp ERR! stack at ChildProcess. (C:\Users\USERNAME\Programming\FRT_Labelling\Form-Recognizer-Toolkit\SampleCode\LabelingUX\Client\node_modules@mapbox\node-pre-gyp\lib\util\compile.js:89:23)
npm ERR! node-pre-gyp ERR! stack at ChildProcess.emit (node:events:513:28)
npm ERR! node-pre-gyp ERR! stack at maybeClose (node:internal/child_process:1091:16)
npm ERR! node-pre-gyp ERR! stack at ChildProcess._handle.onexit (node:internal/child_process:302:5)
npm ERR! node-pre-gyp ERR! System Windows_NT 10.0.19044
npm ERR! node-pre-gyp ERR! command "C:\Program Files\nodejs\node.exe" "C:\Users\USERNAME\Programming\FRT_Labelling\Form-Recognizer-Toolkit\SampleCode\LabelingUX\Client\node_modules\@mapbox\node-pre-gyp\bin\node-pre-gyp" "install" "--fallback-to-build" "--update-binary"
npm ERR! node-pre-gyp ERR! cwd C:\Users\USERNAME\Programming\FRT_Labelling\Form-Recognizer-Toolkit\SampleCode\LabelingUX\Client\node_modules\canvas
npm ERR! node-pre-gyp ERR! node -v v18.12.1
npm ERR! node-pre-gyp ERR! node-pre-gyp -v v1.0.10
npm ERR! node-pre-gyp ERR! not ok

npm ERR! A complete log of this run can be found in:
npm ERR! C:\Users\USERNAME\AppData\Local\npm-cache_logs\2023-01-03T14_09_50_142Z-debug-0.log

C:\Users\USERNAME\Programming\FRT_Labelling\Form-Recognizer-Toolkit\SampleCode\LabelingUX\Client>

Unable to see hover popup on Labeling UX

I've downloaded the latest version as of now on my computer and followed instructions of README.MD. When I run the project, am not able to see any popup when hovered over PDF file for which OCR is already done. Instead, it simply shows yellow boxes.

Can someone please help to fix this?

Resource not found sample_rag_langchain.ipynb

Hi,
I'm trying to run the /SampleCode/Python/sample_rag_langchain.ipynb file. But in the middle when it tries to connect to Azure search it throws the following error.

Does anyone has an idea about fixing this? My Azure Search Instance, OpenAi Instance and Document Intelligence instance are in the same region(East US) and in the same resource group.

Compile fails "Compiled with problems: / Module build failed / TypeError: Cannot read properties of undefined (reading 'replace')

Compiled with problems:

ERROR in ./src/index.scss (./node_modules/css-loader/dist/cjs.js??ruleSet[1].rules[1].oneOf[7].use[1]!./node_modules/postcss-loader/dist/cjs.js??ruleSet[1].rules[1].oneOf[7].use[2]!./node_modules/resolve-url-loader/index.js??ruleSet[1].rules[1].oneOf[7].use[3]!./node_modules/sass-loader/dist/cjs.js??ruleSet[1].rules[1].oneOf[7].use[4]!./src/index.scss)

Module Error (from ./node_modules/sass-loader/dist/cjs.js):
Node Sass does not yet support your current environment: Windows 64-bit with Unsupported runtime (108)
For more information on which environments are supported please see:
https://github.com/sass/node-sass/releases/tag/v7.0.3

ERROR in ./src/index.scss (./node_modules/css-loader/dist/cjs.js??ruleSet[1].rules[1].oneOf[7].use[1]!./node_modules/postcss-loader/dist/cjs.js??ruleSet[1].rules[1].oneOf[7].use[2]!./node_modules/resolve-url-loader/index.js??ruleSet[1].rules[1].oneOf[7].use[3]!./node_modules/sass-loader/dist/cjs.js??ruleSet[1].rules[1].oneOf[7].use[4]!./src/index.scss)

Module build failed (from ./node_modules/resolve-url-loader/index.js):
TypeError: Cannot read properties of undefined (reading 'replace')

Failed to access local files

ERROR in src/view/components/messageModal/messageModal.tsx:39:17

TS2769: No overload matches this call.
  Overload 1 of 2, '(props: IBaseModalProps | Readonly<IBaseModalProps>): Modal', gave the following error.
    Type '{ isOpen: boolean; isLoading: boolean | undefined; header: Element; body: ReactNode; footer: Element; buttonGroup: Element; loadingText: string | undefined; width: number | undefined; onClose: (() => void) | undefined; onDismiss: (() => void) | undefined; }' is not assignable to type 'IntrinsicAttributes & IntrinsicClassAttributes<Modal> & Readonly<IBaseModalProps> & Readonly<...>'.
      Property 'isOpen' does not exist on type 'IntrinsicAttributes & IntrinsicClassAttributes<Modal> & Readonly<IBaseModalProps> & Readonly<...>'.
  Overload 2 of 2, '(props: IBaseModalProps, context: any): Modal', gave the following error.
    Type '{ isOpen: boolean; isLoading: boolean | undefined; header: Element; body: ReactNode; footer: Element; buttonGroup: Element; loadingText: string | undefined; width: number | undefined; onClose: (() => void) | undefined; onDismiss: (() => void) | undefined; }' is not assignable to type 'IntrinsicAttributes & IntrinsicClassAttributes<Modal> & Readonly<IBaseModalProps> & Readonly<...>'.
      Property 'isOpen' does not exist on type 'IntrinsicAttributes & IntrinsicClassAttributes<Modal> & Readonly<IBaseModalProps> & Readonly<...>'.
    37 |         return (
    38 |             <Modal
  > 39 |                 isOpen={isOpen}

....

[1]
[1] ERROR in src/view/containers/labelPane/labelPane.tsx:3:61
[1] TS2305: Module '"@fluentui/react"' has no exported member 'Text'.
[1]     1 | import * as React from "react";
[1]     2 | import { connect, ConnectedProps } from "react-redux";
[1]   > 3 | import { Stack, IconButton, DirectionalHint, IButtonStyles, Text } from "@fluentui/react";
[1]       |                                                             ^^^^
[1]     4 |
[1]     5 | import { ApplicationState } from "store";
[1]     6 | import {
[1]

Unable to pass the file_path

I can't pass documents from my local desktop and unable to provide Url or file_path

windows can run this?