Retaining focus on InputText with on-screen keyboard about imgui

[masranber]

Thank you for the quick responses.

@PathogenDavid Thank you for clearing up active vs focus. Sounds like the approach I'm trying won't work regardless of whether I'm using the focused or active ID.

@ocornut Since you think the 2nd context may be the easiest solution, I'll focus my efforts there.

To clear up the windowing backend, the main application is using GLFW on Linux. However, imgui has no knowledge of the actual backend because the UI is behind its own interface for platform, compatibility, and app architecture/design reasons. So all the display and input event IO binding between the interface and imgui is custom. Hopefully I can use that to my advantage with multiple contexts.

I'll give the two context approach a shot and update later with my findings. Thanks guys.

[masranber]

I got the on-screen/virtual keyboard working with multiple contexts with imgui_impl_opengl3.cpp. Wasn't too hard. Biggest challenge is making sure the correct context is active at all times, which I added my own context switching API for.

The UI uses its own ImGui IO bindings, not one of the included implementations (GLFW, etc..).

Also this code may not be 100% functional as-is since it's been stripped down.

Context Switching

The purpose of adding proper context switching rather than just calling SetCurrentContext() is to ensure the previous context is restored. Without it, it's easy to start rendering elements in the wrong context, sending IO events to the wrong context, etc... especially once you start mixing in nested context switches.

I also added an RAII wrapper to automatically manage the context switch in the current scope (less verbose and bug prone than doing it manually with Push/PopContextSwitch).

imgui_context_extensions.h

namespace ImGui {

void PushContextSwitch(ImGuiContext *newCtx);
void PopContextSwitch();

class ScopedContextSwitch {
public:
    explicit ScopedContextSwitch(ImGuiContext *newCtx);
    ScopedContextSwitch(const ScopedContextSwitch& copy) = delete;
    ScopedContextSwitch& operator=(const ScopedContextSwitch& copy) = delete;
    ScopedContextSwitch(ScopedContextSwitch&& move) = delete;
    ScopedContextSwitch& operator=(ScopedContextSwitch&& move) = delete;
    ~ScopedContextSwitch();
};

} // namespace ImGui

imgui_context_extensions.cpp

static std::stack<ImGuiContext*> CONTEXT_STACK;

namespace ImGui {

void PushContextSwitch(ImGuiContext *newCtx) {
    CONTEXT_STACK.push(ImGui::GetCurrentContext()); // save current context
    ImGui::SetCurrentContext(newCtx); // perform context switch
}

void PopContextSwitch() {
    if(CONTEXT_STACK.empty()) return;
    ImGui::SetCurrentContext(CONTEXT_STACK.top()); // restore previous context
    CONTEXT_STACK.pop();
}

ScopedContextSwitch::ScopedContextSwitch(ImGuiContext *newCtx) {
    ImGui::PushContextSwitch(newCtx);
}

ScopedContextSwitch::~ScopedContextSwitch() {
    ImGui::PopContextSwitch();
}

} // namespace ImGui

Setting Up Contexts

mainCtx is a pointer to the main ImGuiContext instance.
oskCtx is a pointer to the on-screen keyboard ImGuiContext instance.

The rest of the code assumes these contexts have been properly initialized beforehand. Something important I figured out is the backend must be initialized for all contexts. So all functions like ImGui_ImplOpenGL3_Init() must be called with each context active in order to work correctly. Same goes for the NewFrame and Render functions. Not sure if this applies to other backends, or just OpenGL.

Handling IO

// Called by the main application when a mouse button is pressed or released
void ApplicationGUI::OnMouseClick(int x, int y, int button, bool down) {
    ImGui::ScopedContextSwitch outer_scope(oskCtx);
    ImGuiIO &oskIO = ImGui::GetIO();
    oskIO.AddMouseButtonEvent(button, down);

    // Only send mouse click to main context if the keyboard didn't use it
    // Effectively z-layering IO events
    // I.e. user clicked in void around keyboard
    if(!oskIO.WantCaptureMouse) {
        ImGui::ScopedContextSwitch inner_scope(mainCtx);
        ImGui::GetIO().AddMouseButtonEvent(button, down);
    }
}

// Called by the main application when the mouse cursor position changes
void ApplicationGUI::OnMouseMove(int x, int y) {
    ImGui::ScopedContextSwitch outer_scope(oskCtx);
    ImGuiIO &oskIO = ImGui::GetIO();
    oskIO.AddMousePosEvent(x, y);

    // Same z-layering as OnMouseClick(), with keyboard having priority
    if(!oskIO.WantCaptureMouse) {
        ImGui::ScopedContextSwitch inner_scope(mainCtx);
        ImGui::GetIO().AddMousePosEvent(x, y);
    }
}

Render

// Called by main application when the GUI should render
void ApplicationGUI::RenderFrame(int vpW, int vpH) {
    // ---------------------- Main UI -----------------------
    ImGui::ScopedContextSwitch cur_scope(mainCtx);
    ImGui_ImplOpenGL3_NewFrame();
    ImGui::NewFrame();

    ImGuiIO &mainIO = ImGui::GetIO();
    // <calc IO DeltaTime, DisplaySize> if using custom backend

    if(ImGui::Begin("Main Window", nullptr, ImGuiWindowFlags_AlwaysAutoResize)) {
        static char text1[100] = "some text";
        ImGui::InputText("InputText1", text1, IM_ARRAYSIZE(text1));
        static char text2[100] = "more text";
        ImGui::InputText("InputText2", text2, IM_ARRAYSIZE(text2));
    }
    ImGui::End(); // Main Window

    ImGui::EndFrame();
    ImGui::Render();
    ImGui_ImplOpenGL3_RenderDrawData(ImGui::GetDrawData());

    // --------------- On-Screen Keyboard -----------------
    // IMPORTANT: Keyboard drawn AFTER main UI to ensure it's always rendered ON TOP
    ImGui::SetCurrentContext(oskCtx); // main ScopedContextSwitch still in scope so can set context directly
    ImGui_ImplOpenGL3_NewFrame();
    ImGui::NewFrame();

    ImGuiIO &oskIO = ImGui::GetIO();
    // <calc IO DeltaTime, DisplaySize> if using custom backend

    if(mainIO.WantTextInput) {
        ImGui::SetNextWindowPos(ImVec2(oskIO.DisplaySize.x * 0.5f, oskIO.DisplaySize.y * 0.5f), ImGuiCond_Always, ImVec2(0.5f,0.5f));
        ShowKeyboard(mainCtx);
    }

    ImGui::EndFrame();
    ImGui::Render();
    ImGui_ImplOpenGL3_RenderDrawData(ImGui::GetDrawData());
}

And here's an example keyboard with a letter key and backspace key:

bool ShowKeyboard(ImGuiContext *mainCtx) {
    bool keyboard_open = true;
    if(ImGui::Begin("Keyboard", &keyboard_open, ImGuiWindowFlags_AlwaysAutoResize | ImGuiWindowFlags_NoCollapse | ImGuiWindowFlags_NoResize)) {
        const ImVec2 keySize(40, 0);
        if(ImGui::Button("a", keySize)) {
            ImGui::ScopedContextSwitch cur_scope(mainCtx); // The main context must be active to push IO events
            ImGui::GetIO().AddInputCharacter('a');
        }
        ImGui::SameLine();
        if(ImGui::Button(ICON_FA_DELETE_LEFT, keySize)) {
            ImGui::ScopedContextSwitch cur_scope(mainCtx);
            ImGui::GetIO().AddKeyEvent(ImGuiKey_Backspace, true); // emulate backspace pressed
            ImGui::GetIO().AddKeyEvent(ImGuiKey_Backspace, false);
        }
    }
    ImGui::End(); // Keyboard window

    // Unfocus the InputText to avoid keyboard reopening due to io.WantTextInput still being true
    if(!keyboard_open) {
        ImGui::ScopedContextSwitch cur_scope(mainCtx);
        ImGui::GetIO().AddKeyEvent(ImGuiKey_Enter, true);
        ImGui::GetIO().AddKeyEvent(ImGuiKey_Enter, false);
    }
    return keyboard_open;
}

Hopefully others find this information useful.

[PathogenDavid]

I don't have an immediate solution for you, but I can answer some of your questions.

(Also I'm pretty sure there's another on-screen keyboard thread besides #1418 but I can't seem to find it.)

Figure out how to save/restore focus to the InputText. Am I doing something wrong in my example??

Dear ImGui has two different concepts you can call focus. There's focused items and active items. The focused item is used by navigation. The active item is used to determine which widget receives input.

You're controlling the focused item, but the active item is what you actually care about. (As noted by this comment, ImGui::SetKeyboardFocusHere is somewhat misleadingly named as it's indirectly controlling the active item.)

Unfortunately this won't be as simple as just using GetActiveID/ActiveIdWindow/SetActiveID instead as that upsets the internals of ImGui::ButtonBehavior. (I didn't have time to look into why.)

Even if that worked, I think you'd likely be losing out on button repeat behavior which seems desirable for an on-screen keyboard.

// doesn't seem to return the focused window, GetFocusedWindow() doesn't exist

You're looking for ImGui::GetCurrentContext()->NavWindow

However I'm not entirely sure how well the GLFW/OpenGL backends work with multiple contexts.

The GLFW backend is known to be problematic when used with multiple contexts.

[ocornut]

Intuitively I feel like using a second context may be the easiest solution.
The issue for GLFW backend we can look into fixing - it is a fairly simple change, if just only noisy.
OP suggested they have their own windowing backend in the template, but refer to GLFW later on so unsure if they would be directly affected. Most other backends don't have this issue which can be handled on app side. EDIT: SDL backend has same issue, could likewise be fixed.

[ocornut]

Note that the GLFW/SDL backends will likely need the multi-context fix, even though there's a slight chance it might work without.

[PathogenDavid]

Glad you got your multi-context solution working, and thanks for sharing your solution!

One thing you might add is disabling the mouse via AddMousePosEvent(-FLT_MAX, -FLT_MAX); for the mainCtx when oskCtx is owns it. I'm pretty sure that right now if you had a button bordering the edge of the OSK and hovered over it on the way to the OSK it would appear stuck in the hovered state.