Coder Social home page Coder Social logo

cuhacking / atlas Goto Github PK

View Code? Open in Web Editor NEW
15.0 15.0 1.0 523 KB

The map project - codename Atlas

Kotlin 74.05% Ruby 0.42% Swift 16.39% HTML 0.41% TypeScript 5.11% JavaScript 2.84% Dockerfile 0.79%
kotlin kotlin-multiplatform mapbox typescript

atlas's People

Contributors

dellisd avatar ghaskins99 avatar jtibs18 avatar safa-howaid avatar willmahler avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

Forkers

dragonriser

atlas's Issues

Dockerize and deploy web server

Write a Dockerfile to generate a docker image for the web server.

Create actions to deploy the image to the dev server for testing.

Setup project base

Create the project, it will be a gradle project with the following modules:

  • Common/Shared Kotlin Multiplatform module
  • Android app module
  • Mapbox Multiplatform module (will be split into its own project later on in development)
  • Non-gradle iOS app project
  • A web frontent module

Add GeoJSON adapter for SQLDelight database

Overview

The json column in the feature table will hold GeoJson strings, specifically for Features. In our code, we would like to handle these json strings as objects instead, and using SQLDelight's custom column types we can have these strings automatically convert themselves to and from their string representations so that we can use them as objects in our code.

In this case, the json column should be updated to store values as Feature objects from Spatial K.

Task Description

  1. Update the SQLDelight table definition to adapt the json colum to the Feature class from Spatial K.
  2. Implement a corresponding ColumnAdapter in common code.
  3. Write unit tests to test the adapter on the JVM

Validate data against schema instead of by parsing

Use a JSON schema to validate map data when loading rather than trying to parse it. Using a schema can provide better error messages to indicate why the data wasn't parsed.

In addition, the command line component of the server can include a mode to test the data without actually starting the server.
e.g. ./server.jar --test data.json could validate the data and then terminate.

Requires #85 to be complete first.

Set up Mapbox iOS SDK

Set up integration with the Mapbox SDK for both the iOS app project (through XCode) as well as for the Kotlin Multiplatform bindings (mapbox module). You can use the mmapp project as a reference on how to set up the Mapbox SDK with the Kotlin Multiplatform module.

When setting up the Mapbox SDK in the iOS app, do not include a Mapbox API key in the Info.plist file. The API key should instead be set programatically and will come from a build config field that will be configured as part of #2.

Construct basic search UI for iOS

Build the UI for the search feature, and then following the MVVM pattern, connect it with the existing search database.

For this task, the UI should be built using Swift UI.

The end result should look something like this:
image
where the empty space in the background will be filled by the map.

Design Kotlin DSL abstraction of Mapbox style expressions

Overview

Mapbox's layers use style expressions to control how data appears on the map. These expressions allow the appearance of features on the map to be driven by data from the features themselves, or from the map itself.

In order to effectively make use of this capability in a Kotlin Multiplatform project, we want to create bindings for these expressions so that they can be created in Kotlin but then be used in the SDK for each respective platform.

Technical details

Our goal is to create a Kotlin DSL for building these expressions that might look something along the lines of this:

paint {
    fillColor("#00ffff")
    fillOpacity(0.5f)
}

The result (return value) of this building would be something that could then be used in other Multiplatform bindings for the layers themselves that would then be translated/adapted to the platform-specific SDK calls to construct those expressions at run-time.

Abstraction vs Implementation

The DSL is an abstraction. Each platform implements the Mapbox style spec in a very different way and our job is to unify them into a single interface (the DSL) for building them.

Android

On Android, the style expressions are built using static factory methods that return an Expression object. When adding them to a layer, they are a passed to a variable argument function. This is the most similar to what we are looking to create, however we can make use of more Kotlin syntactic sugar to make the interface more appealing while also providing better IDE autocomplete (those static factory methods are a pain for the IDE to find because there's no scoping on them, unlike with Kotlin DSLs).

The Kotlin abstraction will need to map to those factory methods in the platform-specific implementation details.

JS

In JS, the mapbox style spec is represented using a object where property names are mapped to values.

iOS

iOS is once again different in implementing the style specification. It uses a "standard" NSExpression class for modelling style expressions which makes use of many strings instead of array-based operators like in Android and on JS.

The Kotlin DSL will need to map to these expressions.

End Goal

The goal isn't to create a 1-to-1 mapping of the Kotlin DSL to each platform's implementation since they are all too different for this to be possible. Instead, we are trying to create an intermediate representation of the expressions that can then be converted to the platform-specific structures at runtime.

The mmapp project is a good reference for this since the data source for GeoJSON data was implemented on all three platforms. Each one was implemented in a very different way.

Add map to Android app

Add a Mapbox MapView to the Android app's main activity centered around the Carleton University campus. This should be implemented in the android module as a plain native Android view.

Add a layer with a random feature visible on the map.

This map will be used to test implementations of Mapbox classes and functions in future tasks.

Finalize App UI Designs

Finalize the UI designs for the Android, iOS, and Web apps. Mockups should be included and should cover the colours, shapes, typography, and icons to be included in the final design.

The scope of this task is to design the UI components themselves. The design of the map itself will be covered by another task.

Fix search autocomplete

Current entire words need to be matched for results to show up in the results list. The search should autocomplete as the user is typing.

Display Map Data From Database

Now that data is being downloaded and stored in the database, display it on the map. This will replace the test data currently declared in-code.

Finalize Initial Data Schema

Finalize an initial schema for the map data for use in development and testing.

The schema should be represented as a JSON schema that can be used to validate the data set.

Construct basic search UI for Android

Build the UI for the search feature, and then following the MVVM pattern, connect it with the existing search database.

For this task, there is the option between using Android's layout system, or making use of Jetpack Compose (which is very new).

The end result should look something like this:
image
where the empty space in the background will be filled by the map.

Add map to web app

Add a map that fills the entire screen centered around the Carleton University campus. This should be implemented through React and in TypeScript.

Add a layer with a random feature visible on the map.

This map will be used to test implementations of Mapbox classes and functions in future tasks.

This task is very similar to cuhacking/atlas-editor#1, but the difference will be where the Mapbox access token comes from. In that task, it came from a .env file, but here it will come from the compiled Kotlin/JS code.

TypeScript Interop

Before starting, make sure to follow the build instructions in the README. You will need to make sure you run gradlew build otherwise the compiled JavaScript won't be visible and yarn will complain.

Because we are using TypeScript, we need TypeScript definitions for the compiled Kotlin/JS code. In Kotlin 1.4 the compiler will automatically generate the definitions for us, but we're stuck on 1.3.70 for the time being... so we have to write them manually. This was already done in 4016cb4, but nonetheless the way you import these definitions is a little strange since Kotlin namespaces all of the compiled code.

You can see this in index.tsx under the web directory, like this:

import { com } from "Atlas-common";
import BuildKonfig = com.cuhacking.atlas.common.BuildKonfig;

// Use it here
BuildKonfig.MAPBOX_KEY

This constant was compiled from Kotlin code, how cool is that??

Add abstract Mapbox Layer

Create the base of a class hierarchy for representing Layers in Mapbox. Attributes that are common to all types of layers include an id, source, filter (an expression), layout (expressions), paint (expressions), minzoom and maxzoom, and others listed in the Mapbox docs.

This should be an abstract, expected class in the mapbox module's commonMain sourceset.

Provide actual implementations for Android and JS, as with the GeoJsonSource.

Implement Mapbox GeoJSon data source for iOS

Note that this task is almost identical to #19, but specifically for iOS. This task will make use of the same common expect declarations as #19.

Overview

Using the common GeoJson data source in the mapbox module's common code, provide the implementation for iOS. The implementations can be based off of the implementations in mmapp.

Technical Details

When the common declaration of a GeoJsonSource is imported to platform-specific view (i.e. to use in the view to add to the Map), we would like to be able to treat the source object as though it were already a platform-specific instance of the data source. In other words, we would like to be able to say that our source is a MGLShapeSource when referenced in iOS code. This is the same as in #19. However, there are currently limitations in how Kotlin is allowed to subclass Objective-C classes, so this isn't possible.

Instead, in the actual implementation of the GeoJsonSource, you have to keep an instance of a MGLMapSource and then update that instead. This instance will then be exposed to the iOS code instead.

So instead of:

let source = GeoJsonSource(...)
mapView.style?.addSource(source)

we'll have:

let source = GeoJsonSource(...)
mapView.style?.addSource(source.internalSource)

although the name "internalSource" can be changed.

Task Instructions

  1. Create actual implementations of this class for iOS.
  2. Create an instance of a GeoJsonSource with some random feature and then add them to the map that was added in #16.

Enable strict mode on mapbox module

Since we want to treat this module as a library (that will eventually be split off), enable strict mode which is a new feature of Kotlin 1.4 to ensure the module follows library writing best-practices.

Blocked by #25

Write data server

Write a simple server that will serve a single static JSON file.
The endpoint must also specify a Last-Modified header containing the last time the file was updated.

The server should be written with Ktor.

Implement full-text search in database

Using FTS4, implement full-text search in the database. FTS requires a virtual table that should mirror the existing "Features" table created in an earlier task. Any item that is inserted into the main features table should also be inserted into the virtual table.

Not all values from the original table need to be mirrored. For instance, the json values from the original table are basically useless for FTS.

Create a "search" query that returns all rows that match a given query string, as well as another query to clear the virtual table.

Testing

Test the FTS implementation on the JVM.

  1. Test that inserting a row into the original features table also inserts it into the virtual table.
  2. Test that clearing the virtual table works
  3. Insert a few values into the virtual table where some of the fields contain some text and test that the "search" query returns rows that match any or all of the text in those fields. Example: If a feature has a common_name of "River Building", the "search" query with "river" as an argument should return that row.

Future Considerations

We may choose to use FTS5 in the future, but this will require more development work for a later time.

Upgrade to Kotlin 1.4.10

Upgrade is important since the project is currently broken for anyone developing on Windows because of KT-40834 which was fixed in 1.4.10

Blocked by:

  • SQLDelight
  • Spatial K

Finalize Map Design

This task covers the design/aesthetic of the map itself. This includes the colours, typography, and icons of various features that would be situated on a map including both outdoor and indoor features.

Add declarations for Database instances

We'll need some way to create instances of the Database to use. expect and actual won't actually be the best for accomplishing most of this because of a key difference required for the Android Driver to work.

My recommendation is to implement an extension function similar to this one which will help create a database instance given an instance of a driver. Then, we'll focus on creating the driver instances separately.

Create an expected function called provideDbDriver that returns a SqlDriver and then provide actual implementations in each platform sourceset. Then add a global lateinit var called database that will store the result of provideDbDriver later on.

Essentially, at some point during the application's initialization on each platform, an instance of the database will need to be created. The entire application will have **only one instance ** of the database to share throughout the entire codebase, i.e. the database will be a singleton. There are much nicer ways of handling this including many fancy libraries and frameworks, but we won't touch those for now because a) everyone on the team needs more experience first and b) there are no perfect solutions for Kotlin Multiplatform yet, but it's mostly a.

Android Driver

The key difference for the Android Driver that I mentioned above is that it needs a Context. For the time being, create a global lateinit var called appContext that will store a Context and use that to create the driver. This solution is far from pretty, but it's the best we can do for the time being.

In Android apps, there is an Application class that can handle initialization of the app process. Create a subclass of the Application class called AtlasApp and override the onCreate to initialize the previously created database variable. I'll leave the magic spell to initialize appContext as a learning exercise for you.

iOS Driver

This driver doesn't require anything special, but the database still needs to be initialized when the app process starts similar to the Android app.

JS Driver

Since SQLDelight isn't publishing artifacts for the JS driver yet, it's hard for us to test. I recommend trying to build the SQLDelight driver locally as described here, but if that doesn't work just create an implementation of the expected functions with a TODO

Upgrade to Kotlin 1.4.0

Now that Kotlin 1.4 is stable, upgrade to it and update all required dependencies.

This is currently blocked by:

  • SQLDelight Maven Central (note, this 1.4.0 does not correspond to Kotlin's 1.4.0)
  • Spatial K Sonatype Nexus (Snapshots)
  • Ktor Download

Once these dependencies are upgraded appropriately:

  • Update Kotlin version
  • Update dependency versions
  • Apply any necessary changes to coding practices and gradle configs for Kotlin 1.4 (mostly in JS and Native targets).

Set up Mapbox Android SDK

Set up integration with the Mapbox SDK for both the Android app (android module) as well as for the Kotlin Multiplatform Mapbox bindings (mapbox module).

For the Android app module, a Mapbox API Key should be read in from the local.properties file and included as a build config field in the common module so that it can be accessed from each platform frontend. (You can use BuildKonfig for this). The mapbox module does not require an API key or any extra setup.

Write iOS target tests for caching map data

#57 implements a class that handles caching map data, however, no testing have been done to confirm that it functions properly on iOS targets.

The tests must ensure that the JSON data can be both written to and read from the cache.

Get Last-Modified time from server

Extend the remote api class from #21 to include another function that makes a HEAD request to the same endpoint, then return the parsed value (using a date/time library) of the Last-Modified header of the response.

Add SQLDelight to project

Add the SQLDelight library and plugin to the project.

The plugin will need to be applied to the root project (in the project folder's build.gradle.kts file).
The library will need to be added to the common module (in that folder's build.gradle.kts).

Look under the buildSrc folder to see how dependencies are being managed in the project.

Upgrade project iOS target to 14.0

Version needs to be updated in Versions.kt and in Podfile (possibly elsewhere, need to double check when this gets worked on).
Also need to check for and resolve any build/compatibility issues... Hopefully these are minimal, if there are any at all ๐Ÿคž.
Note: As of this issue's creation date, current target is 13.7.

Implement Map Interactions

Tapping/clicking on features on the map should open a popup of some kind which shows some details of the selected feature.

The design aspects of this will need to be confirmed with the @cuhacking/design team.

Add map to iOS app

Add a map to the iOS app's main screen centered around the Carleton University campus.

This map should be added using SwiftUI. Mapbox has a good tutorial on integrating the Mapbox MapView with Swift UI. The Annotations section can be omitted, however you should add a layer to the map with a random feature visible.

The Mapbox access token should be provided programmatically by importing it from the Common framework.

This is blocked by #3

Implement Mapbox GeoJson data source for Android and JS

Overview

Implement a GeoJson data source in the mapbox module's common code and then provide implementations for Android and JS. The implementations can be based off of the implementations in mmapp.

Technical Details

In common Kotlin code, we can declare a GeoJsonSource like so: val source = GeoJsonSource(data) .

When this common declaration is imported to platform-specific view (i.e. to use in the view to add to the Map), we would like to be able to treat the source object as though it were already a platform-specific instance of the data source. In other words, we would like to be able to say that our source is a com.mapbox.mapboxsdk.style.sources.GeoJsonSource when referenced in Android code, and that our source matches the contract for a data source for use in the mapbox-gl-js library. (i.e. has type: "geojson" and data properties). Examples of how this works can be found in the mmapp project.

Task Instructions

  1. Create the common definition of the GeoJsonSource.
  2. Create actual implementations of this class for Android and JS.
  3. Create an instance of a GeoJsonSource with some random feature and then add them to the map that was added in #15 and #17.

Remove test data and use FeatureApi

The map and search bar are currently fed separate test data. This should be removed and replaced with the actual source of data: FeatureApi. The API will need to be made use of to download test data from the development server, and then stored in the DataCache and Database.

The application (on all platforms) must check and/or download data at launch. This will involve launching a coroutine that calls the appropriate API methods and ensures that any downloading or other I/O operations happens in the "background". Some care may need to be paid attention to in order to make sure that the correct CoroutineScope is used on each platform to avoid memory leaks.

Once the launching function^ is written, it should be as simple as calling it in the correct place on each platform. On Android this can be initiated in the onCreate() method of the AtlasApplication class. On iOS this needs to be handled in the UIApplicationDelegate protocol. In JS, this needs to be called when the application is created (possibly in App.tsx?).

Cache map data

Along with #21 which downloads map data and inserts it into the database to be indexed for searching, we also want to cache the full set of data so that it can be loaded directly onto the map. To do this, on Android and iOS it should be saved as a file on-device, and in JS it can just be kept in memory (it only needs to exist as long as the web page is open).

Technical Details

Create a new class using the expect/actual mechanism that contains a function that will handle reading and writing JSON data.

  • For Android and iOS: write data to a file, and also read data from that file
  • For JS: save (write) the JSON data in a variable, and return (read) the value of that variable
    There are existing Kotlin Multiplatform libraries for doing file I/O on Android and iOS that can be used for this part of the task, but it's entirely optional. The actual implementations can simply call the native methods to read and write files.

This class should also track the last time the cached data was updated. This will be used to determine whether or not the cache should be updated by comparing the cache's time with the Last-Modified header of the remote API response. This will require adding a dependency to a date/time library, such as this one.

Finally, update the data repository class to write data to this cache when the data is updated.

Add logic for downloading map data into database

Overview

Using a Ktor client, implement a GET request that downloads some JSON data.

Technical Details

This function should be encapsulated in a class. The class should have two constructor properties: one for the Database, and one for an HttpClient. These properties will be supplied by other parts of the code later on.

The server endpoint that the request will be made to should be configured via the local.properties file similar to how our mapbox keys are currently set up. Add a string field to the buildkonfig setup to read in some url. For the scope of this task, it doesn't matter what this url is since we won't be testing with an actual server.

The returned JSON data will be a FeatureCollection following the GeoJson standard. You can use the Spatial K library to convert this JSON text into an object.

Using the queries and table created in #13, insert each Feature into the table by mapping each Spatial K feature to a database Feature. Here's how the features from the FeatureCollection map to the database for now:

Name Mapping
fid Get from Feature properties
common_name Use name property from props
secondary_name Set to null for now
type Use type property from props
building Use building property from props, or null if not available
floor Use floor property from props, or null if not available
search_tags Use the name property from props for now
json Convert the Feature back to a string. (This will be done by an adapter once #18 is complete.)

I recommend creating a separate function for this. An extension function would be good for this purpose.

Testing

Using Ktor's mock client(s), write tests to ensure that data is correctly downloaded and inserted into the database. I recommend waiting until #18 is finished so that you can reuse some of the database test code.

Create map data SQLite table and queries

This database will be built using SQLDelight and will be a SQLite database.

Table

The map data table will require the following columns:

Name Type Description
fid integer (not null) Unique ID for each feature
common_name text (not null) A common name for a feature
secondary_name text An (optional) secondary name for a feature
type text (not null) A type classifier for each feature
building text Two letter code for the building this feature is in
floor text A code that indicates what floor of a building
search_tags text Other text that can be indexed for searching
json text (not null) The JSON object for this feature

Queries

The following queries will need to be added:

  • Insert feature
  • Get a feature by its fid value
  • Get all features
  • Delete all features

Add Ktor client to project

Add the Ktor client library to the project. Be sure to follow the instructions for adding the library to a Multiplatform project.

Look under the buildSrc folder to see how dependencies are being managed in the project.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.