Uploader Application

[spinozist]

• Property Geocoder (see line 222 of app/uploader.py in HOUSEATL_UPLOAD)

• Map CSVs to database collections/fields (see app/mapping.py in HOUSEATL_UPLOAD) using geocoded location as unique property id to join to other collections

[hermanjohn2]

@spinozist I found the issue. The reason there are only 13 properties is because only 13 of them have totalSubsidizedUnits > 0 . totalSubsidizedUnits is calculated from the subsidies low_income_units. This dummy data only has 13 properties that fit that category. I removed the filter and all 1000 showed up with the now joined FundingSources.

I am going to push where I am at and pick it back up in the morning. Will fix the populate for the subsidy route once I start entering the actual data because the relationships will be set up at that point. Right now they are referencing the "id" field so I am running a hacky loop. Still works perfectly but will be refactored to have a nice populate on it soon.

[spinozist]

Excellent! Proceed as you are. I’m going to put a “check-in” on the calendar for tomorrow morning to assess progress, but if you reach the next big benchmark before then, please let me know. The plan is to get the API up and running with the newest data by early next week. I think we are well on track to meet that deadline but keeping a tight schedule will help insure it. Many thanks, Erik Woodworth ***@***.*** From: John Herman ***@***.***> Date: Tuesday, January 4, 2022 at 5:02 PM To: atlregional/houseatl_api ***@***.***> Cc: Erik Woodworth ***@***.***>, Mention ***@***.***> Subject: Re: [atlregional/houseatl_api] Uploader Application (Issue #2) @spinozist<https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fspinozist&data=04%7C01%7Cewoodworth%40atlantaregional.org%7Cc082f62a3f7b4f899a3308d9cfd646e9%7C1efd81f59e5345999ec376e7b5dbdf81%7C0%7C0%7C637769341514147891%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=r0t7klz2i0TPYIsItHeNQKzozpmwzLHGSTkoP%2Fy0F5U%3D&reserved=0> I found the issue. The reason there are only 13 properties is because only 13 of them have totalSubsidizedUnits > 0 . totalSubsidizedUnits is calculated from the subsidies low_income_units. This dummy data only has 13 properties that fit that category. I removed the filter and all 1000 showed up with the now joined FundingSources. I am going to push where I am at and pick it back up in the morning. Will fix the populate for the subsidy route once I start entering the actual data because the relationships will be set up at that point. Right now they are referencing the "id" field so I am running a hacky loop. Still works perfectly but will be refactored to have a nice populate on it soon. — Reply to this email directly, view it on GitHub<https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fatlregional%2Fhouseatl_api%2Fissues%2F2%23issuecomment-1005234877&data=04%7C01%7Cewoodworth%40atlantaregional.org%7Cc082f62a3f7b4f899a3308d9cfd646e9%7C1efd81f59e5345999ec376e7b5dbdf81%7C0%7C0%7C637769341514147891%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=Zy08oReDwUNNDMIUoYiy42Vw06VABNp5FzkWQB5Eb%2Bk%3D&reserved=0>, or unsubscribe<https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAFQICRO5XTPC7GGCZ6BRK2DUUN4AFANCNFSM5KTDIN5A&data=04%7C01%7Cewoodworth%40atlantaregional.org%7Cc082f62a3f7b4f899a3308d9cfd646e9%7C1efd81f59e5345999ec376e7b5dbdf81%7C0%7C0%7C637769341514304129%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=VYZVvwLy0LUGdR5ApTAQFN8%2Brv0sVZC61kR3jZK1Qpk%3D&reserved=0>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[hermanjohn2]

@spinozist now working front to back with a small sample of NHPD data. I'll try to get all Agencies working by our meeting tomorrow. Should be able to run on every file by later in the day on Monday for sure.... maybe tomorrow depending on how far I get today.

[spinozist]

Excellent! See you tomorrow morning. Erik Woodworth ***@***.*** From: John Herman ***@***.***> Date: Thursday, January 6, 2022 at 3:28 PM To: atlregional/houseatl_api ***@***.***> Cc: Erik Woodworth ***@***.***>, Mention ***@***.***> Subject: Re: [atlregional/houseatl_api] Uploader Application (Issue #2) @spinozist<https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fspinozist&data=04%7C01%7Cewoodworth%40atlantaregional.org%7C23733aae8ae743768b3108d9d15b7ccd%7C1efd81f59e5345999ec376e7b5dbdf81%7C0%7C0%7C637771013147713646%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=M%2FtVojqtf34AfS8QeidBolnzl3ZVdfPC%2BcyYNrHMLOs%3D&reserved=0> now working front to back with a small sample of NHPD data. I'll try to get all Agencies working by our meeting tomorrow. Should be able to run on every file by later in the day on Monday for sure.... maybe tomorrow depending on how far I get today. — Reply to this email directly, view it on GitHub<https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fatlregional%2Fhouseatl_api%2Fissues%2F2%23issuecomment-1006947616&data=04%7C01%7Cewoodworth%40atlantaregional.org%7C23733aae8ae743768b3108d9d15b7ccd%7C1efd81f59e5345999ec376e7b5dbdf81%7C0%7C0%7C637771013147713646%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=4VLyEUSLHLn%2B34FI6uLc62lkONE8%2FxtDjQXnxjMXrnw%3D&reserved=0>, or unsubscribe<https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAFQICRMCY6HKR4FJQYTPDNTUUYCP7ANCNFSM5KTDIN5A&data=04%7C01%7Cewoodworth%40atlantaregional.org%7C23733aae8ae743768b3108d9d15b7ccd%7C1efd81f59e5345999ec376e7b5dbdf81%7C0%7C0%7C637771013147713646%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=WCg7Z7SqneaZmZGyAtJHM2t8jMh2deFvQwdbxvXxjv8%3D&reserved=0>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[hermanjohn2]

@spinozist we should be able to run this Updater on all xlsx files by tomorrow. I have a couple of minor tweaks/testing I am going to be doing today but I think I might a have a few questions related to the data. After those are addressed we should be good to go.

Can we get together tomorrow?

[hermanjohn2]

@spinozist I hate to do this but is there any way I can re-upload the data to the database? I found a bug with dates and I want to include agency id's and upload id's when creating a subsidy. Will help with deduplicating process.

[spinozist]

Let’s meet (briefly, I promise) at 3 to discuss the plan before implementing. I’m also showing the populated dashboard to Mike at 2 so want to wait until after that. On Jan 13, 2022, at 12:26 PM, John Herman ***@***.***> wrote:  @spinozist<https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fspinozist&data=04%7C01%7Cewoodworth%40atlantaregional.org%7Cf80df46853334d4151b408d9d6c24604%7C1efd81f59e5345999ec376e7b5dbdf81%7C0%7C0%7C637776952178606595%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=y7JGo30VTG4%2FVv0r0LehI8n3MkuQzAY5ODObgkpACHA%3D&reserved=0> I hate to do this but is there any way I can re-upload the data to the database? I found a bug with dates and I want to include agency id's and upload id's when creating a subsidy. Will help with deduplicating process. — Reply to this email directly, view it on GitHub<https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fatlregional%2Fhouseatl_api%2Fissues%2F2%23issuecomment-1012396995&data=04%7C01%7Cewoodworth%40atlantaregional.org%7Cf80df46853334d4151b408d9d6c24604%7C1efd81f59e5345999ec376e7b5dbdf81%7C0%7C0%7C637776952178606595%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=TibfJ86P%2FdhJK6C3ihLtgS%2B28el%2BgUD4d%2Bd4Qr7Ynwk%3D&reserved=0>, or unsubscribe<https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAFQICRJM3JI5KGV7PI72OHDUV4KO3ANCNFSM5KTDIN5A&data=04%7C01%7Cewoodworth%40atlantaregional.org%7Cf80df46853334d4151b408d9d6c24604%7C1efd81f59e5345999ec376e7b5dbdf81%7C0%7C0%7C637776952178606595%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=P2HU3lSACI2XoE%2FhQczTiJDqGMF70gXeGJQk55uYNv0%3D&reserved=0>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[hermanjohn2]

Sounds good. John On Thu, Jan 13, 2022 at 1:43 PM Erik Woodworth ***@***.***> wrote:

Let’s meet (briefly, I promise) at 3 to discuss the plan before implementing. I’m also showing the populated dashboard to Mike at 2 so want to wait until after that. On Jan 13, 2022, at 12:26 PM, John Herman ***@***.***> wrote:  @spinozist< https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fspinozist&data=04%7C01%7Cewoodworth%40atlantaregional.org%7Cf80df46853334d4151b408d9d6c24604%7C1efd81f59e5345999ec376e7b5dbdf81%7C0%7C0%7C637776952178606595%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=y7JGo30VTG4%2FVv0r0LehI8n3MkuQzAY5ODObgkpACHA%3D&reserved=0> I hate to do this but is there any way I can re-upload the data to the database? I found a bug with dates and I want to include agency id's and upload id's when creating a subsidy. Will help with deduplicating process. — Reply to this email directly, view it on GitHub< https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fatlregional%2Fhouseatl_api%2Fissues%2F2%23issuecomment-1012396995&data=04%7C01%7Cewoodworth%40atlantaregional.org%7Cf80df46853334d4151b408d9d6c24604%7C1efd81f59e5345999ec376e7b5dbdf81%7C0%7C0%7C637776952178606595%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=TibfJ86P%2FdhJK6C3ihLtgS%2B28el%2BgUD4d%2Bd4Qr7Ynwk%3D&reserved=0>, or unsubscribe< https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAFQICRJM3JI5KGV7PI72OHDUV4KO3ANCNFSM5KTDIN5A&data=04%7C01%7Cewoodworth%40atlantaregional.org%7Cf80df46853334d4151b408d9d6c24604%7C1efd81f59e5345999ec376e7b5dbdf81%7C0%7C0%7C637776952178606595%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=P2HU3lSACI2XoE%2FhQczTiJDqGMF70gXeGJQk55uYNv0%3D&reserved=0>. You are receiving this because you were mentioned.Message ID: ***@***.***> — Reply to this email directly, view it on GitHub <#2 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ANYYK24K2MLBOUVOKIXUV7TUV4MODANCNFSM5KTDIN5A> . You are receiving this because you were assigned.Message ID: ***@***.***>

-- Sent from my iPhone