Responsible AI Ukraine Presentation Discussion

[himanshunaidu]

Greetings RAI members,
@NaturalStupidlty @mariana0412 @lordwordsword
So this is just a small thread I am creating to have any kinds of discussions necessary for the RAI Project Presentation on December 19th.

As mentioned by Professor Anat, the RAI program expects you to take the lead on the presentation preparation, since you would be the ones presenting it. But we would be happy to help you prepare for it every step of the way. Feel free to decide where and how much you'd like us to contribute towards.

Based on the high level discussion we had today regarding the discussion, I see the following main points in the presentation (not necessarily in order):

1. Project Background and Motivation: This is something that we would be happy to elaborate upon, as emphasized by Professor Anat. We can probably have a light talk about the the background of the whole Transportation Data Equity Initiative that this project is a part of. Feel free to ask us whatever questions you have on this. You can choose how much to focus on this.

2. Individual Responsibilities: This is something you should be talking about in detail. The RAI program wants to know what all did you get to do, anything you got to take the lead on, what all you got to learn from it, what would you have liked to do better, etc.

Also if you had any qualms with the project, you could be free to express that in the presentation. I won't turn a blind eye to the fact that there are a lot of ways this project could have been executed better.
(I do realize we also did not set up any explicit avenue for getting timely feedback from you, mostly because I didn't want to add even more meetings to our already packed schedule. Maybe I can just get feedback from all of you in the coming weeks.)

3. Demo: While I don't see us releasing this app on TestFlight anytime soon, we could have a demo for all our existing work. We will need to test this app rigorously in the last few days before the presentation.

4. Future directions: This is of course, something that Professor Anat and me could help you on. So feel free to ask us whatever questions you have. Example, we could talk about any plans we have for things open beta access (although we haven't really decided on this as of yet).

These are of course, just some pointers, so feel free to add on to this as much as possible.

[himanshunaidu]

Presentation Deliverables

Confirmed

1. User Login:
   Users who have an account in TDEI Workspaces will be able to login to the application.

2. Segmentation Post-Processing (Point Features):
   We will be identifying point features.
   For linear features (such as sidewalks, although this is dependent on which segmentation model is used), we will identify the center lines as well as the width.

3. Data Structuring and API Integration:
   All the geometries will be formatted to have a consistent co-ordinate system (WGS-84) and valid format for the Workspaces API.

4. User Election (AnnotationView) Improvements:
   Currently, the user can edit the tagging for a specific class in the AnnotationView.

Tentative

1. Segmentation Model Improvements
   We unfortunately had to revert from the usage of the DeepLabv3 CityScapes model to the ESPNetv2 PASCAL, as we underestimated the load DeepLabv3 had on the CPU for real-time segmentation. We will try to integrate a better model.

2. Segmentation Post-Processing (Linear Features)
   While we have the tools to set up the linear feature identification, due to the segmentation model issues and delays in post-processing logic, we may have issues implementing this on time.

3. Support for non-Pro iOS devices
   We are working on integrating monocular depth estimation. It is still work-in-progress, but if it is successfully integrated, non-Pro devices will also be able to support the app, although the depth estimation accuracy will be lower.

De-Scoped

1. Instance-based Segmentation:
   Currently our instance segmentation model is not complete, so we will observe only one instance per segment. Thus, the user cannot select one of the multiple instances to edit tagging. However, we will try to refactor the application in such a way that it can seamlessly support multiple instances of the same class, in the future.

2. Support for KartaView
   While we will not be able to support KartaView API to send representative images, we will explore this API for future work.

3. Segmentation Model Fine-Tuning
   Since we do not have the correct training data for our use case, we cannot proceed with fine-tuning the segmentation model.