Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
133 commits
Select commit Hold shift + click to select a range
6779347
feat(street): add street project draft fixture
ofr1tz Oct 29, 2024
40cdc66
feat: add new street project
Gigaszi Oct 29, 2024
33bc5ec
feat: add tutorial to street project
Gigaszi Oct 29, 2024
ec59ee0
refactor: move process_mapillary.py to utils
Gigaszi Oct 30, 2024
78f28df
feat: add functions to process mapillary data
Gigaszi Oct 30, 2024
3a7ac21
refactor: removed unused imports and classes
Gigaszi Oct 30, 2024
8e5c55c
feat: add function to convert input geojson to polygon
Gigaszi Oct 30, 2024
dd4e220
tests: add unittests for process_mapillary
Gigaszi Oct 30, 2024
b274af3
fix: wrong indention and patch in test_process_mapillary
Gigaszi Oct 30, 2024
7ec92ee
feat: add filtering and handling of mapillary response
Gigaszi Oct 30, 2024
97e552c
tests: add tests for filtering and handling of mapillary response
Gigaszi Oct 30, 2024
53dfda5
refactor: use mapillary incredentials in variables
Gigaszi Nov 11, 2024
481030c
refactor: reformat file process_mapillary.py
Gigaszi Nov 11, 2024
1187561
feat: add geometry to StreetTask and refactoring
Gigaszi Nov 11, 2024
0bef2e0
tests(street): use simpler geometry and add filter parameters to test…
Gigaszi Nov 11, 2024
789b027
refactor: tests for mapillary processing and street project
Gigaszi Nov 11, 2024
c1f1a79
fix: add missing parameter
Gigaszi Nov 11, 2024
a12a803
tests(street): add integration test for street project
Gigaszi Nov 11, 2024
2fb3bc7
tests(street): add street.json for testing of street project
Gigaszi Nov 11, 2024
14ff237
fix: test after refactoring
Gigaszi Nov 11, 2024
80ecf28
feat: add functions for spatial sampling and tests for it
Gigaszi Nov 11, 2024
984d821
feat: use filtering in street project
Gigaszi Nov 14, 2024
4ab528d
feat: add spatial sampling to street project
Gigaszi Nov 18, 2024
f7fc5f3
feat: add size restriction for too many images
Gigaszi Nov 18, 2024
f11f75e
breaking: change geom column in postgres to allow all geometry types
Gigaszi Nov 18, 2024
98f82c9
feat: save location of image as geometry
Gigaszi Nov 18, 2024
6a1d15e
feat: use spatial filter on downloaded images
Gigaszi Nov 18, 2024
05f1449
fix: adapt mapillary token entry in example env
ofr1tz Nov 28, 2024
263d738
fix: do not return failed_rows and do not use functions on empty df
Gigaszi Nov 28, 2024
5ac3620
Merge branch 'feature/street' of https://github.com/mapswipe/python-m…
Gigaszi Nov 28, 2024
df17d3c
fix: testing for removed funcionality
Gigaszi Nov 28, 2024
9f2ac81
style: code formatting
ofr1tz Nov 28, 2024
8593a3c
fix: fixed tests and removed return of failed rows for download from …
Gigaszi Nov 28, 2024
81d44ed
Merge branch 'feature/street' of https://github.com/mapswipe/python-m…
Gigaszi Nov 28, 2024
9503a7d
fix: tests for removed failing rows
Gigaszi Nov 28, 2024
c0016e2
style: fix flake8 errors and isort
ofr1tz Nov 28, 2024
2f3d523
style: isort
ofr1tz Nov 28, 2024
6ad174f
build: add requirements
ofr1tz Nov 28, 2024
315d239
build: add dummy mapillary key to github workflow
ofr1tz Nov 28, 2024
2b88cf7
fix: use os.getenv instead of os.environ
ofr1tz Nov 28, 2024
24c7eec
test: rename class and correct comments
ofr1tz Nov 28, 2024
7f46e80
fix: remove left overs of failed rows
Gigaszi Dec 3, 2024
652e885
Merge branch 'feature/street' of https://github.com/mapswipe/python-m…
Gigaszi Dec 3, 2024
bfa0818
fix: unittests for tile download
Gigaszi Dec 3, 2024
816b1e6
fix: add condition to raise valueerror if no features are found in aoi
Gigaszi Dec 3, 2024
f287b3a
fix: use mapillary api key secret in github workflow
ofr1tz Dec 3, 2024
020178d
fix: use square brackets with os.environ
ofr1tz Dec 3, 2024
dcaab3d
fix: add mapillary key to docker compose
ofr1tz Dec 3, 2024
8128488
fix: allow tasks with point geom in postgres
ofr1tz Dec 3, 2024
74064f8
fix: change geometry type in tasks table in initdb
Gigaszi Dec 3, 2024
f712340
Merge branch 'feature/street' of https://github.com/mapswipe/python-m…
Gigaszi Dec 3, 2024
729a4be
fix: change to all geometries in tasks table in setup db
Gigaszi Dec 3, 2024
994f398
fix: use lower case in setup db
Gigaszi Dec 3, 2024
1c1a6ba
feat: add integration test for tutorials
Gigaszi Dec 4, 2024
dadca06
fix: delete tutorial draft at end of test
Gigaszi Dec 4, 2024
e1ec359
feat(manager-dashboard): street project definitions
ofr1tz Dec 10, 2024
0188ba1
feat(manager-dashboard): invisible tile server input in street projects
ofr1tz Dec 10, 2024
8059804
feat(manager-dashboard): add custom options preview and project aoi g…
ofr1tz Dec 10, 2024
c4f43c9
feat(manager-dashboard): set default groupSize for street
ofr1tz Dec 10, 2024
f9ae768
feat(manager-dashboard): add banner to alert that street projects are…
ofr1tz Dec 10, 2024
b463566
feat(manager-dashboard): add street specific inputs (wip)
ofr1tz Dec 10, 2024
57dc509
feat(django): add new project types to project type model
ofr1tz Dec 11, 2024
26c822c
fix: sort imports
ofr1tz Dec 11, 2024
4787dd4
fix: add new project types to schema.graphql
ofr1tz Dec 11, 2024
78c4908
feat(manager-dashboard): add validation for organizationId
ofr1tz Dec 11, 2024
905dcde
feat(manager-dashboard): add mapillary image filter inputs (wip)
ofr1tz Dec 11, 2024
6bc241a
Merge pull request #980 from mapswipe/feature/street
ofr1tz Dec 12, 2024
8c9d774
WIP: add tutorial for street project
Gigaszi Dec 12, 2024
e90ebe5
feat(manager-dashboard): complete mapillary image filter inputs
ofr1tz Dec 12, 2024
5ad603e
fix: use clip of image locations to aoi
Gigaszi Dec 16, 2024
775088f
Merge pull request #984 from mapswipe/feature/clip-street
ofr1tz Dec 16, 2024
fb6610e
feat: add filter by creator_id
Gigaszi Dec 17, 2024
c3b58aa
Merge pull request #985 from mapswipe/street-project-creator-id
ofr1tz Dec 17, 2024
88de7c2
feat(manager-dashboard): add creator id input to mapillary image filters
ofr1tz Dec 17, 2024
8593cc8
feat(manager-dashboard): adjust value interface
ofr1tz Dec 17, 2024
0f54062
fix: exception handling when no elements are matched with filter
Gigaszi Dec 17, 2024
2a25bd8
Merge pull request #986 from mapswipe/street-project-exception-handling
ofr1tz Dec 17, 2024
508edf8
Merge pull request #987 from mapswipe/feature/manager-dashboard/street
ofr1tz Dec 19, 2024
4811d94
fix(manager-dashboard): correctly set isPano for street projects
ofr1tz Jan 16, 2025
d3b3f3d
feat: sequences are now in direction of travel
Gigaszi Jan 20, 2025
417a5fa
fix: just use on dataframe to save memory
Gigaszi Jan 20, 2025
3f09ae7
feat: add option to randomize order of images
Gigaszi Jan 20, 2025
b9e0a49
fix: change test to match new code
Gigaszi Jan 20, 2025
b8b2c14
feat(manager-dashboard): Add randomizeOrder input for new Street proj…
ofr1tz Jan 22, 2025
e2486d9
fix: remove artifacts from debugging
Gigaszi Jan 27, 2025
bc2bcd5
feat: use ProcessPoolExecutor for download_and_process_tile
Gigaszi Jan 27, 2025
c3dca08
fix(street): get_image_metadata() error handling
ofr1tz Jan 27, 2025
cb12624
fix: tasks are only randomized if randomize_order is True
Gigaszi Jan 27, 2025
6e78598
fix: remove articat from debugging
Gigaszi Jan 27, 2025
538c763
fix(street): adjust tests
ofr1tz Jan 27, 2025
d4dd6a0
refactor: extract functions for improved testing
Gigaszi Jan 28, 2025
87d65cb
fix: adapt tests to new multiprocessing
Gigaszi Jan 28, 2025
41d6413
fix: remove debugging artifact
Gigaszi Jan 28, 2025
23b654e
fix: remove debugging artifact
Gigaszi Jan 28, 2025
7899bfb
refactor: use logger.info instead of logger.exception for missing values
Gigaszi Jan 28, 2025
7ff2688
fix: requesting the same tile multiple times for multipolygon
Gigaszi Jan 28, 2025
4d7790b
fix: remove unnecessary check for empty dataframe
Gigaszi Jan 30, 2025
916d4f7
feat: drop duplicated images at exact same location
Gigaszi Jan 30, 2025
e62d69b
Merge pull request #988 from mapswipe/fix/street-pano-filter
ofr1tz Jan 30, 2025
e91373d
Merge branch 'main' into improve-street-project
Gigaszi Feb 3, 2025
50cb94f
fix: randomize after spatial sampling
Gigaszi Feb 3, 2025
8b37284
fix: reassign downloaded metadata
Gigaszi Feb 3, 2025
7847b9a
Merge branch 'dev' into fix/mapillary-image-error-handling
ofr1tz Feb 4, 2025
3bdc562
feat: images are in sequential order if randomizeOrder is false or none
Gigaszi Feb 4, 2025
c23b45f
Merge pull request #993 from mapswipe/fix/tiles-processing
ofr1tz Feb 4, 2025
2d41751
Merge pull request #989 from mapswipe/improve-street-project
ofr1tz Feb 4, 2025
624127b
Merge branch 'dev' into feat/manager-dashboard/randomize-order-street
ofr1tz Feb 4, 2025
2a458b5
Merge pull request #990 from mapswipe/feat/manager-dashboard/randomiz…
ofr1tz Feb 6, 2025
c6d5141
Merge branch 'main' into improve-ressource-usage
Gigaszi Feb 6, 2025
d8a5833
Merge branch 'dev' into improve-ressource-usage
Gigaszi Feb 6, 2025
dedd455
feat(manager-dashboard): show custom options input for street tutorials
ofr1tz Feb 6, 2025
3974bf5
Merge pull request #991 from mapswipe/improve-ressource-usage
ofr1tz Feb 6, 2025
f057e6f
feat(manager-dashboard): handle default custom options on project typ…
ofr1tz Feb 6, 2025
3826747
feat(manager-dashboard): hide tileserver input for street tutorials
ofr1tz Feb 6, 2025
0e8b15b
Merge branch 'dev' into fix/mapillary-image-error-handling
Gigaszi Feb 6, 2025
f227899
Merge pull request #992 from mapswipe/fix/mapillary-image-error-handling
ofr1tz Feb 10, 2025
92f0ae9
feat(street-tutorials): add street tutorial sample scenario and instr…
ofr1tz Feb 19, 2025
32ce566
feat(street-tutorials): prepare scenario pages input for street tutorial
ofr1tz Feb 19, 2025
5890915
Merge branch 'dev' into street-tutorial
Gigaszi Feb 20, 2025
dfb3743
feat: change structure of street tutorial to match those of other pro…
Gigaszi Feb 20, 2025
550df7a
fix: working integration test for tutorial creation
Gigaszi Feb 20, 2025
5237ce0
fix: naming in integration test
Gigaszi Feb 20, 2025
e2efb18
refactor: change name of test_tutorial to test_tutorial_street
Gigaszi Feb 20, 2025
f3ae549
feat(street-tutorial): correct reference value in sample data
ofr1tz Feb 20, 2025
c27e228
feat(street-tutorial): ensure that custom options are written to tuto…
ofr1tz Feb 20, 2025
65555a8
Merge pull request #983 from mapswipe/street-tutorial
ofr1tz Feb 20, 2025
514ccb8
Merge pull request #1006 from mapswipe/feature/manager-dashboard/stre…
ofr1tz Feb 20, 2025
64553ef
fix: remove duplicates after spatial sampling
Gigaszi Feb 24, 2025
2e2152c
feat: add unittest for each tutorial
Gigaszi Mar 3, 2025
809f0f8
fix: handle FutureWarning when concating dfs
Gigaszi Mar 3, 2025
a7319e4
fix: check if column exists before using astype
Gigaszi Mar 3, 2025
6d89ff6
Merge pull request #1007 from mapswipe/fix-drop-duplicates
ofr1tz Mar 4, 2025
1c0e540
Merge pull request #1009 from mapswipe/add-tutorial-tests
ofr1tz Mar 4, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/actions.yml
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,7 @@ jobs:
POSTGRES_DB: postgres
OSMCHA_API_KEY: ${{ secrets.OSMCHA_API_KEY }}
DJANGO_SECRET_KEY: test-django-secret-key
MAPILLARY_API_KEY: ${{ secrets.MAPILLARY_API_KEY }}
COMPOSE_FILE: ../docker-compose.yaml:../docker-compose-ci.yaml
run: |
docker compose run --rm mapswipe_workers_creation python -m unittest discover --verbose --start-directory tests/unittests/
Expand Down
5 changes: 4 additions & 1 deletion django/apps/existing_database/models.py
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,9 @@ class Type(models.IntegerChoices):
FOOTPRINT = 2, "Validate"
CHANGE_DETECTION = 3, "Compare"
COMPLETENESS = 4, "Completeness"
MEDIA = 5, "Media"
DIGITIZATION = 6, "Digitization"
STREET = 7, "Street"

project_id = models.CharField(primary_key=True, max_length=999)
created = models.DateTimeField(blank=True, null=True)
Expand Down Expand Up @@ -127,7 +130,7 @@ class Task(Model):
project = models.ForeignKey(Project, models.DO_NOTHING, related_name="+")
group_id = models.CharField(max_length=999)
task_id = models.CharField(max_length=999)
geom = gis_models.MultiPolygonField(blank=True, null=True)
geom = gis_models.GeometryField(blank=True, null=True)
# Database uses JSON instead of JSONB (not supported by django)
project_type_specifics = models.TextField(blank=True, null=True)

Expand Down
3 changes: 3 additions & 0 deletions django/schema.graphql
Original file line number Diff line number Diff line change
Expand Up @@ -97,6 +97,9 @@ enum ProjectTypeEnum {
FOOTPRINT
CHANGE_DETECTION
COMPLETENESS
MEDIA
DIGITIZATION
STREET
}

type ProjectTypeSwipeStatsType {
Expand Down
1 change: 1 addition & 0 deletions docker-compose.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,7 @@ x-mapswipe-workers: &base_mapswipe_workers
SLACK_CHANNEL: '${SLACK_CHANNEL}'
SENTRY_DSN: '${SENTRY_DSN}'
OSMCHA_API_KEY: '${OSMCHA_API_KEY}'
MAPILLARY_API_KEY: '${MAPILLARY_API_KEY}'
depends_on:
- postgres
volumes:
Expand Down
3 changes: 3 additions & 0 deletions example.env
Original file line number Diff line number Diff line change
Expand Up @@ -75,3 +75,6 @@ COMMUNITY_DASHBOARD_GRAPHQL_ENDPOINT=https://api.example.com/graphql/
COMMUNITY_DASHBOARD_SENTRY_DSN=
COMMUNITY_DASHBOARD_SENTRY_TRACES_SAMPLE_RATE=
COMMUNITY_DASHBOARD_MAPSWIPE_WEBSITE=https://mapswipe.org

# Mapillary
MAPILLARY_API_KEY=
2 changes: 2 additions & 0 deletions manager-dashboard/app/Base/configs/projectTypes.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ import {
PROJECT_TYPE_BUILD_AREA,
PROJECT_TYPE_FOOTPRINT,
PROJECT_TYPE_CHANGE_DETECTION,
PROJECT_TYPE_STREET,
PROJECT_TYPE_COMPLETENESS,
} from '#utils/common';

Expand All @@ -15,6 +16,7 @@ const mapswipeProjectTypeOptions: {
{ value: PROJECT_TYPE_BUILD_AREA, label: 'Find' },
{ value: PROJECT_TYPE_FOOTPRINT, label: 'Validate' },
{ value: PROJECT_TYPE_CHANGE_DETECTION, label: 'Compare' },
{ value: PROJECT_TYPE_STREET, label: 'Street' },
{ value: PROJECT_TYPE_COMPLETENESS, label: 'Completeness' },
];

Expand Down
7 changes: 6 additions & 1 deletion manager-dashboard/app/Base/styles.css
Original file line number Diff line number Diff line change
Expand Up @@ -122,9 +122,14 @@ p {
--line-height-relaxed: 1.625;
--line-height-loose: 2;


--shadow-card: 0 2px 4px -2px var(--color-shadow);

--duration-transition-medium: .2s;

--color-background-hover-light: rgba(0, 0, 0, .04);
--width-calendar-date: 2.4rem;

--opacity-watermark: 0.3;
--color-text-disabled: rgba(0, 0, 0, .3);

}
68 changes: 68 additions & 0 deletions manager-dashboard/app/components/Calendar/CalendarDate/index.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
import React from 'react';
import { _cs } from '@togglecorp/fujs';

import RawButton, { Props as RawButtonProps } from '../../RawButton';
import { ymdToDateString, typedMemo } from '../../../utils/common.tsx';

import styles from './styles.css';

export interface Props {
className?: string;
year: number;
month: number;
date: number;
currentYear: number;
currentMonth: number;
activeDate?: string;
currentDate: number;
onClick?: (year: number, month: number, date: number) => void;
elementRef?: RawButtonProps<undefined>['elementRef'];
ghost?: boolean;
}

function CalendarDate(props: Props) {
const {
className,
year,
month,
date,
currentYear,
currentMonth,
currentDate,
onClick,
elementRef,
activeDate,
ghost,
} = props;

const handleClick = React.useCallback(() => {
if (onClick) {
onClick(year, month, date);
}
}, [year, month, date, onClick]);

const dateString = ymdToDateString(year, month, date);

return (
<RawButton
elementRef={elementRef}
name={date}
className={_cs(
styles.date,
year === currentYear
&& month === currentMonth
&& currentDate === date
&& styles.today,
dateString === activeDate && styles.active,
ghost && styles.ghost,
className,
)}
onClick={handleClick}
>
{date}
</RawButton>

);
}

export default typedMemo(CalendarDate);
25 changes: 25 additions & 0 deletions manager-dashboard/app/components/Calendar/CalendarDate/styles.css
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
.date {
border-radius: 50%;
width: var(--width-calendar-date);
height: var(--width-calendar-date);

&.today {
color: var(--color-accent);
font-weight: var(--font-weight-bold);
}

&:hover {
background-color: var(--color-background-hover-light);
}

&.active {
background-color: var(--color-accent);
color: var(--color-text-on-dark);
pointer-events: none;
}

&.ghost {
opacity: 0.5;
}
}

Loading
Loading