[!TODO] Read the "Database first approach with DotNet Core" article in Obsidian before continuing to work on this project.
This repository contains the code for the Judge and Tournament Registration websites for NoVA North Odyssey of the Mind.
| Folder | Project Name | .NET Version | Purpose |
|---|---|---|---|
| / | EFCoreToolReverseEngineeringTest | .NET 10.0 | ==TBD== |
| / | JudgeRegistrationRazor | .NET 10.0 | ==TBD== |
| /OdysseyRegistration/ | docker-compose | N/A | Docker Compose project (.dcproj) * Also builds the OdysseyRegistrationWebApi project. |
| /OdysseyRegistration/ | init | N/A | SQL Server initialization files for the Docker Compose project |
| /OdysseyRegistration/ | Odyssey.Database | SQL Server 2019 (==how can you tell which version?==) | Database management project (.sqlproj) |
| /OdysseyRegistration/ | OdysseyCoreMvc | .NET 10.0 | Unused (I think): Core MVC framework |
| /OdysseyRegistration/ | OdysseyMvc4 | .NET Framework 4.8 | MVC framework 4 This is what is running in production today (01/01/2025). |
| /OdysseyRegistration/ | OdysseyMvc4.Tests | .NET Framework 4.8 | MVC framework 4 tests |
| /OdysseyRegistration/ | OdysseyMvc2023 | .NET Framework 4.8 | MVC framework 2023, ==I think unused since it's .NET Framework== |
| /OdysseyRegistration/ | OdysseyMvc2024 | .NET 10.0 | ==Is this the current project or is it the 2023 project? I think it's this one.== |
| /OdysseyRegistration/ | OdysseyRegistrationWebApi | .NET 10.0 | Web API for registration |
| /OdysseyRegistration/ | UpdateProblemSynopsesForRegistration | .NET Framework 4.8 | Update problem synopses, ==Need to figure out what this does and if it's worth keeping== |
The OdysseyMvc4 (original) and OdysseyMvc2023 (current? I think it's really OdysseyMvc2024 build on .NET) projects are currently built using ASP.NET MVC version 4 on .NET Framework 4.8.
- ✅ BaseRegistrationController - Complete
- ✅ HomeController - Complete
- ✅ JudgesRegistrationController - Complete
- ❓TournamentRegistrationController - Stopped reviewing on line 144.
- Update (all) the projects to use ASP.NET (Core) 10.0 or later.
- Document how I added the SQL Project to Visual Studio and how to modify, use, and deploy it.
- Rolled back to EF 4.4 to make sure everything worked. See if the code works as-is with EF 6.x.
- Create Elmah test DB
- Determine what all the projects in this solution are for.
- Upgrade the Odyssey.Database project to SQL Server 2022.
- Clean up the "Purpose" column in the Projects table above.
- Add logging to all projects!
- Make all "Return to the Home Page" buttons return to the home page at the current base URL, not hard-coded to a specific Odyssey Registration home page.
- Move docker-compose.dcproj and its associated files into its own subdirectory.
- Set up automated SQL Server Backups for Odyssey Registration.
- 09/29/2024: Read "Get started with Entity Framework Core and an existing database in minutes - Quick Start Guide":
- https://x.com/ErikEJ/status/1740635086742069720, 2:25 AM · Dec 29, 2023.
- This can produce Mermaid diagrams, as well.
- 11/09/2024: Add sweetalert2 to replace alerts.
- A beautiful, responsive, customizable, accessible (WAI-ARIA) replacement for JavaScript's popup boxes. Zero dependencies.
- 11/17/2024: Mandatory: Add the membership name and number to the tournament registration email sent to the coach.
- 11/17/2024: Preferable: Populate the membership name and number from the schools table into the tournament registration table at the time of registration.
- 01/01/2025: Roll back anything in the .NET updates that are incompatible with the .NET Framework / ASP.NET v4 version of the website just to get onto .NET (Core) and be finished with .NET Framework once and for all.
- 01/01/2025: This includes the changes I made to the Odyssey database schema.
- 01/01/2025: Once the .NET (not Framework) site is in production, then re-add updates/changes that were checked in today.
- 01/01/2025: Convert the Run the SQL Server database in a Docker container (manually) section into a Dockerfile that can be built instead of step-by-step PowerShell cmdlets.
- 04/06/2025: Add Humanizer: meets all your .NET needs for manipulating and displaying strings, enums, dates, times, timespans, numbers and quantities.
- 04/06/2025: Figure out why the
sqlserver.configuratordocker container doesn't run once thesqlservercontainer is healthy.
- Make sure that the file
sa_password.txtexists in theOdysseyRegistration\OdysseyRegistrationdirectory.- Also make sure that it is not a directory.
- It should only contain the sa account's password in plain text.
- You should NOT need to modify the following files to set the SQL Server sa account password:
OdysseyRegistration\init.sqlOdysseyRegistration\init\init.sql- The password should automatically be read from
OdysseyRegistration\sa_password.txt. - See the first numbered item in this list, above.
- Update
WebmasterEmailPasswordwith the correct value in the following files:OdysseyRegistration\novanorth-prod.sqlOdysseyRegistration\init\novanorth-prod.sql- This value can be found in the Config table of the Production Odyssey SQL Server database hosted on WinHost's server.
- Steps (or link to steps): TBD
The OdysseyMvc2024 project uses ASP.NET Core User Secrets to store the database connection string securely:
Setup (already completed):
# Navigate to the project directory
cd OdysseyRegistration\OdysseyMvc2024
# Initialize User Secrets (adds UserSecretsId to csproj)
dotnet user-secrets init
# Set the connection string (password must match sa_password.txt)
dotnet user-secrets set "ConnectionStrings:OdysseyConnection" "Server=localhost;Database=DB_12824_registration;User Id=sa;Password=YOUR_PASSWORD;TrustServerCertificate=True;"
# View stored secrets
dotnet user-secrets list
# Clear all secrets
dotnet user-secrets clearHow it works:
- User Secrets are stored at
%APPDATA%\Microsoft\UserSecrets\{UserSecretsId}\secrets.json - They are automatically loaded in the Development environment
- They override values in
appsettings.json - They are never committed to source control
Important: The password in User Secrets must match the password in OdysseyRegistration/sa_password.txt (used by Docker for SQL Server).
User Secrets are only for development. For production environments (WinHost), use one of these approaches:
ASP.NET Core automatically reads environment variables. In WinHost:
- Log into WinHost Control Panel
- Navigate to Site Manager → IIS Settings → Application Settings
- Add a new application setting:
- Name:
ConnectionStrings:OdysseyConnection(orConnectionStrings__OdysseyConnection) - Value:
Server=s06.winhost.com;Database=DB_12824_registration;User Id=DB_12824_registration_user;Password=YOUR_PROD_PASSWORD;TrustServerCertificate=True;
- Name:
This approach:
- Keeps secrets out of source control
- Can be changed without redeploying
- Is the recommended approach for shared hosting
- Create
appsettings.Production.jsondirectly on the WinHost server (via FTP) - Include the connection string with production password
- Ensure this file is NOT in source control (add to
.gitignore)
{
"ConnectionStrings": {
"OdysseyConnection": "Server=s06.winhost.com;Database=DB_12824_registration;User Id=DB_12824_registration_user;Password=YOUR_PROD_PASSWORD;TrustServerCertificate=True;"
}
}For the legacy OdysseyMvc4 project using .NET Framework:
- Use Web.config transforms (
Web.Release.config) - Or manually edit
Web.configon server after deployment
<connectionStrings>
<add name="OdysseyConnection"
connectionString="Data Source=tcp:s06.winhost.com;Initial Catalog=DB_12824_registration;User ID=DB_12824_registration_user;Password=YOUR_PROD_PASSWORD;Integrated Security=False;"
providerName="System.Data.SqlClient" />
</connectionStrings>If migrating to Azure App Service:
- Create an Azure Key Vault
- Store secrets in Key Vault
- Use managed identity for secure access
- Add the Azure Key Vault configuration provider in
Program.cs
Security Best Practices:
- Never commit passwords to source control
- Use different passwords for development, test, and production
- Rotate passwords periodically
- Use least-privilege database accounts
- Enable TLS/SSL for all database connections
- Following https://learn.microsoft.com/en-us/aspnet/core/security/app-secrets?view=aspnetcore-10.0&tabs=windows.
- This article explains how to manage sensitive data for an ASP.NET Core app on a development machine. Never store passwords or other sensitive data in source code or configuration files. Production secrets shouldn't be used for development or test. Secrets shouldn't be deployed with the app. Production secrets should be accessed through a controlled means like Azure Key Vault. Azure test and production secrets can be stored and protected with the Azure Key Vault configuration provider.
- Use Visual Studio
- In Visual Studio, right-click the project in Solution Explorer, and select Manage User Secrets from the context menu. This gesture adds a UserSecretsId element, populated with a GUID, to the project file.
- If
GenerateAssemblyInfois false/disabled, manually add the UserSecretsIdAttribute in AssemblyInfo.cs. For example:[assembly: UserSecretsId("your_user_secrets_id")]- When manually adding the UserSecretsId attribute to AssemblyInfo.cs, the UserSecretsId value must match the value in the project file.
- HOWEVER, The GenerateAssemblyInfo property controls AssemblyInfo attribute generation for the project. The default value is true.
- Visual Studio's Manage User Secrets gesture opens a
secrets.jsonfile in the text editor. Replace the contents of secrets.json with the key-value pairs to be stored.- NOTE: I got a
secrets.xmlfile for OdysseyMvc4, which is a .NET Framework 4.8 file and not .NET. Maybe that's why there was a difference.
- NOTE: I got a
- Start Docker Desktop for Windows.
- You may need to delete any previous container named
sqlserver.- Otherwise, you may get an error at build time for the
docker-composeproject in VS 2022.
- Otherwise, you may get an error at build time for the
- You may need to delete any previous container named
- Open VS 2022.
- Set the
docker-composeVS 2022 project as the start up project. - Build the
docker-composeproject in the .slnx. This builds two projects:- OdysseyRegistrationWebApi
- docker-compose
- If this is not the first time you've built the project, it should kill and remove the existing containers.
- Right click the
docker-composeproject and selectStart Without Debuggingto start three containers:- odysseyregistrationwebapi
- sqlserver.configurator
- sqlserver
- This resulted in a
There were build errors. Would you like to continue and run the last successful build?dialog box.- Click No.
- The
docker-composeproject build showed the following error in the Error List window in VS 2022:Volume sharing is not enabled. On the Settings screen in Docker Desktop, click Resources -> Shared Drives, and select the drive(s) or folder(s) containing your project files. For more information, please visit - https://aka.ms/DockerToolsTroubleshooting.
- Now start the
sqlserver.configuratorcontainer and the SQL data will be populated in the database.- It will run for a few seconds and exit.
- This is expected.
- You may now connect to the running SQL Server instance using VS 2022, etc., with the following settings:
- Server Name: localhost
- Authentication: SQL Server Authentication
- User name: sa
- Password: <whatever is defined in the
docker-compose\sa_password.txtfile> - Encrypt: Optional (False)
- Connect to database: OdysseyEntities
Warning
The files used to populate the SQL Server database are in the init directory of the docker-compose project.
- The same files in the root directory of the
OdysseyRegistrationproject don't seem to do anything. - For now, make changes to both files until I figure out how to just have one set of files or understand what the difference is between the two.
- The difference may be between dev and prod?
Make sure to copy the web.config file from the OdysseyRegistration directory, i.e., the top-most/root directory, into the root directory of your website at WinHost (the hosting company).
Note
This should be placed in a higher directory than your bin, Content, Views, etc. directories. The directory containing those subdirectories will likely have its own web.config file.
- Make sure that the ASP.NET MVC website directory is set as an application starting point.
- Log into the Winhost Control Panel.
- Navigate to the Odyssey website.
- Navigate to the Sites List ➡️ Site Manager ➡️ Application Starting Point page.
- Make sure that the path to the directory containing the website is registered as an application starting point, e.g.
/registration.
- TODO: Document this procedure.
- Run this from the command prompt or PowerShell
docker container exec sqlserver /opt/mssql-tools18/bin/sqlcmd -S ServerName -U username -P <password> -Q "BACKUP DATABASE [DatabaseName] TO DISK='/Backups/DatabaseName.bak'"docker container exec -it sqlserver /opt/mssql-tools18/bin/sqlcmd -S localhost -U sa -P <password> -Nodocker container exec -it sqlserver /opt/mssql-tools18/bin/sqlcmd -S s06.winhost.com -U DB_12824_registration_user -P <password> -No
use DB_12824_registration
go
select * from config
go
quit
docker container exec -it sqlserver /opt/mssql-tools18/bin/sqlcmd -S s06.winhost.com -U DB_12824_registration_user -P <password> -No
BACKUP DATABASE [DB_12824_registration] TO DISK='/var/opt/mssql/backups/odyssey.bak'
go
Msg 3201, Level 16, State 1, Server S06, Line 1
Cannot open backup device 'D:\DbBackup\/var/opt/mssql/backups/odyssey.bak'. Operating system error 3(The system cannot find the path specified.).
Msg 3013, Level 16, State 1, Server S06, Line 1
BACKUP DATABASE is terminating abnormally.
1> BACKUP DATABASE [DB_12824_registration] TO DISK='odyssey.bak'
2> go
Processed 448 pages for database 'DB_12824_registration', file 'DB_12824_registration_data' on file 1.
Processed 2 pages for database 'DB_12824_registration', file 'DB_12824_registration_log' on file 1.
BACKUP DATABASE successfully processed 450 pages in 0.110 seconds (31.924 MB/sec).Did this get written to the Winhost server in D:\DbBackup? How do I get at that??? How do I delete it??? 5/10/2025: Left off here
Key parameters:
-S: Server name
-E: Use Windows Authentication
-U and -P: For SQL Authentication (username/password)
-Q: Query to execute
Tips:
Make sure the backup directory exists
Ensure your account has sufficient permissions
Replace ServerName and DatabaseName with your values
docker container exec -it sqlserver bashWinHost Control Panel Site: novanorth.org
As of 09/29/2024, this is the configuration of the registration database.
Server Name: s06.winhost.com
Database Name: DB_12824_registration
Version: MS SQL 2008 R2
Database Server: s06.winhost.com
Database User: DB_12824_registration_user
Assigned Quota: 25 MB
Usage: 4 MBConnection String: "Data Source=tcp:s06.winhost.com;Initial Catalog=DB_12824_registration;User ID=DB_12824_registration_user;Password=******;Integrated Security=False;"==How did I create this VS project in the first place?==
- See docker-compose.yml.
- Manually run the
sqlserver.configuratordocker container (from within the Docker Desktop UI) to create the Odyssey login and database.- It should run automatically once the
sqlservercontainer is healthy. I added to TODO to figure out why this isn't running.
- It should run automatically once the
Open a PowerShell prompt.
docker pull mcr.microsoft.com/mssql/server:2022-latest
docker volume create sql-volume
$mssql_sa_password = "" # set this to a strong password
docker run -e 'ACCEPT_EULA=Y' -e "MSSQL_SA_PASSWORD=$mssql_sa_password" -p 1433:1433 --name sqlserver --hostname sqlserver --mount "source=sql-volume,target=/sqldata" -d mcr.microsoft.com/mssql/server:2022-latest
docker exec -it -u 0 sqlserver "bash" # -u 0 lets us log in as root.
chmod 777 /sqldata
docker container exec sqlserver /opt/mssql-tools18/bin/sqlcmd -U sa -P "$mssql_sa_password" -Q "CREATE DATABASE [DB_12824_registration] ON PRIMARY ( NAME = N'DB_12824_registration_data', FILENAME = N'/sqldata/DB_12824_registration_data.mdf' , SIZE = 4160KB , MAXSIZE = 25600KB , FILEGROWTH = 1024KB ) LOG ON ( NAME = N'DB_12824_registration_log', FILENAME = N'/sqldata/DB_12824_registration_log.ldf' , SIZE = 1024KB , MAXSIZE = 1024000KB , FILEGROWTH = 65536KB );"`
docker cp "2022-08-06 - NoVA North Production Database Export Script.sql" sqlserver:/sqldata
docker container exec sqlserver /opt/mssql-tools18/bin/sqlcmd -U sa -P "$mssql_sa_password" -i "/sqldata/2022-08-06 - NoVA North Production Database Export Script.sql"This will create a Mermaid database schema diagram from your SQL Server database.
-
Download latest version: Releases � KarnerTh/mermerd � GitHub
-
Unzip it.
-
Make sure your SQL Server database is up, e.g., in Docker.
-
Run the following command:
Downloads\mermerd_0.4.1_windows_amd64.tar\mermerd -c "sqlserver://sa:********@localhost:1433?database=DB_12824_registration" -s dbo --useAllTables -o OdysseySchema.mmd
-
You will find your file created as OdysseySchema.mmd in the directory where you ran the tool.
docker rm -f sqlserverNote
Using rm -f will remove the container even if it is still running.
- Navigate to Application Starting Point at the hosting company's website.
- Click "Create".
- Enter "/test" as the subdirectory.
- Click "Create".
- You should see a message that the site was created successfully.
- TODO: Add instructions.
- TODO: Add instructions.
- Note that there is an Elmah.xsd file in the root of
OdysseyMvc2023.
- TODO: Add instructions.
Created a new project in the solution named OdysseyRegistrationWebApi
- Right-clicked on the sln and selected "Add, New Project..."
- Selected "ASP.NET Core Web API".
- Used the following options:
- .NET 8.0 (Long Term Support)
- Authentication type: None (I may need to fix this later)
- Configure for HTTPS: checked
- Enable container support: checked
- Do not use top-level statements: unchecked
- Use controllers: checked
- Enlist in .NET Aspire orchestration: unchecked
- I was then prompted with:
⚠️ This solution contains packages with vulnerabilities. Manage NuGet Packages.- Clicked the "Manage NuGet Packages" link.
- I checked "Show only vulnerable".
- jQuery.UI.Combined needed to be updated from 1.10.3 to 1.13.2.
- The only project using it looks like OdysseyMvc4.
- I still got vulnerability warnings in my build output.
- Upgraded jQuery 2.0.3 in OdysseyMvc4.
- jQuery.UI.Combined 1.10.3 OdysseyMvc4.
-
In VS 2022, I right-clicked on the solution and selected Add, Container Orchestrator Support.
- Docker Compose
- Linux Containers
-
I updated the
docker-compose.ymlfile with the contents of one I found for adding a SQL Server 2022 database, modifying it by leveraging Docker Compose secrets.db: image: mcr.microsoft.com/mssql/server:2022-latest environment: MSSQL_SA_PASSWORD_FILE: /run/secrets/sa_password ACCEPT_EULA: "Y" ports: - "1433:1433" -
I followed the instructions here: https://stackoverflow.com/questions/69941444/how-to-have-docker-compose-init-a-sql-server-database.
-
I created
init\init.sqlwith the following content.USE [master]; GO IF NOT EXISTS (SELECT * FROM sys.sql_logins WHERE name = 'vaodyssey') BEGIN CREATE LOGIN [vaodyssey] WITH PASSWORD = '********', CHECK_POLICY = OFF; ALTER SERVER ROLE [sysadmin] ADD MEMBER [vaodyssey]; END GO
-
I copied
OdysseyRegistration\Odyssey.Database\Scripts\2023-12-02 - NoVA North Registration Prod Backup.sqltoOdysseyRegistration\init\novanorth-prod.sql. -
I added a new command to the
docker-compose.ymlfile to create and initialize the Odyssey Registration database:/opt/mssql-tools18/bin/sqlcmd -S sqlserver -U sa -P ${Sa_Password:-********} -d master -i docker-entrypoint-initdb.d/novanorth-prod.sql;
-
Replaced passwords in docker-compose.yml file: https://docs.docker.com/compose/use-secrets/
-
Right-click on the docker-compose project in VS and select "Compose Up".
- TODO: Are any steps missing before this?
- I'm figuring out where I left off and what everything does in the .slnx.
- I believe the OdysseyMvc2023 project is the main project that runs on my desktop against a SQL Server database running in a local Docker container.
- When I built the docker-compose project in the .slnx, it built two projects:
- OdysseyRegistrationWebApi
- docker-compose
- Then I right-click on the docker-compose project and select "Compose Up".
- The sqlserver.configurator-1 container starts and exits.
- The sqlserver-1 container starts and keeps running.
- The odysseyregistrationwebapi-1 container starts and keeps running.
- I can now run SQL Server Management Studio (SSMS) and connect to the SQL Server database running in the sqlserver-1 container.
- The database is fully populated with the data from the
init\novanorth-prod.sqlscript.- TODO: I think that's where the data came from. I need to verify this.
- The database is fully populated with the data from the
- The login for the user in the Odyssey Registration MVC app failed, but otherwise I think it's working on my desktop.
- I updated
OdysseyRegistration\OdysseyMvc2023\Web.configwith Data Source=localhost (i.e., Docker), user=sa, and password=********.
- I updated
- The following code threw an exception in OdysseyRegistration\OdysseyMvc2023\Models\OdysseyRepository.cs on line 355.
Event tournamentInfo = this.tournamentInfo ?? (this.TournamentInfo = Queryable.Where<Event>((IQueryable<Event>)this.context.Events, (Expression<Func<Event, bool>>)(o => o.EventName.StartsWith(this.RegionName) && o.EventName.Contains("Tournament"))).First<Event>());- The
EventNamein theEventsDB table needed to be renamed from "NoVA North Regional Tournament" to "NoVA North and NoVA South Regional Tournament". - This is because I'm using last year's DB data and we were a combined region last year.
- TODO: Consider adding some kind of check for a valid region name to match the number.
- TODO: DEFINITELY add logging for this issue.
- The code continued after this name change!!!
- The web site is running locally!
- Judges and Tournament Registration show as "(Coming Soon!)"
- I commented these lines out and uncommented the lines that allow you to proceed to the registration pages.
- Now both the Judges and Tournament Registration pages are showing Closed.
-
Used EF Core Power Tools to reverse engineer the Odyssey Registration database.
-
To export data from the current SQL Server database as JSON, run a query like the following in SSMS:
SELECT * FROM Schools FOR JSON AUTO
-
Add a link like
entity.HasData(SeedHelper.SeedData<School>("Schools.json"));to themodelBuilder.Entity<School>()code within theOnModelCreatingmethod in theOdysseyContextclass. -
Added EF migrations using the Developer PowerShell for VS 2022 window.
-
Used the following commands:
dotnet ef migrations add InitialCreate dotnet ef database update dotnet ef migrations add UpdateConfigSeedData dotnet ef database update dotnet ef migrations add UpdateSchoolsSeedData dotnet ef database update dotnet ef migrations add UpdateEventSeedData dotnet ef database update dotnet ef migrations add UpdateProblemSeedData dotnet ef database update
- Don't forget that the Problem table Id had to start at 1 where it used to start at 0. So, the code needs to be updated to handle this.
- ==Why is this starting at 1 now? I don't remember.== 😟
- Committed ALL of the modified files to the git repo after removing all passwords.
- Pushed to GitHub.
- Trying to continue making progress, updated this readme file.
- Trying to remember how to get the website up and running.
- I started Docker Desktop in Windows.
- I set the
docker-composeproject as the start up project. - I hit F5 to run the
docker-composeproject.