Commit Graph

26 Commits

Author SHA1 Message Date
nikolajlauridsen
4c39734b51 Merge branch 'release/12.0' into v12/dev
# Conflicts:
#	version.json
2023-07-13 10:38:03 +02:00
Nikolaj Geisle
20a4e475c8 Merge pull request from GHSA-h8wc-r4jh-mg7m
* Don't login after install

* Fail the install if database is not created

---------

Co-authored-by: Zeegaan <nge@umbraco.dk>
Co-authored-by: Nikolaj <nikolajlauridsen@protonmail.ch>
2023-07-13 06:18:34 +02:00
Chad
e784cfa960 Improve logging performance by checking Trace/Debug log level is enabled (#12793) 2023-06-07 11:47:05 +02:00
Bjarke Berg
c662f5a36e Determine correct if we can create a database with current connectionstring (#14030)
* Fix CanForceCreateDatabase method and add some unit tests

* Fixed an old copy/paste error

* A little nitpicking over wording and formatting

---------

Co-authored-by: kjac <kja@umbraco.dk>
2023-03-30 10:19:35 +02:00
Bjarke Berg
6f4b60c76c Updated Nuget packages 2022-12-15 10:36:13 +01:00
Callum Whyte
5ef43cb4f0 Rename StaticServiceProvider namespace Umbraco.Cms.Core.DependencyInjection (#13239) 2022-11-29 12:22:57 +01:00
Ronald Barendse
dca4d0f167 Add runtime mode (BackofficeDevelopment, Development and Production) (#12631)
* Add runtime mode setting

* Only enable Razor runtime compilation in development modes

* Only enable ModelsBuilder generation in development modes

* Fix disabling ModelsBuilder controllers

* Add IRuntimeModeValidationService and IRuntimeModeValidator

* Add JITOptimizerValidator

* Add UmbracoApplicationUrlValidator

* Add UseHttpsValidator

* Add RuntimeMinificationValidator

* Add ModelsBuilderModeValidator

* Remove .NET 6 preview 1 fix for Razor runtime compilation

* Only allow InMemoryAuto in backoffice development mode

* Make runtime mode validators public, so they can be easily removed if required

* Add comment to highlight removing RazorCompileOnBuild, RazorCompileOnPublish and CopyRazorGenerateFilesToPublishDirectory when using ModelsMode InMemoryAuto

* Add documentation

* Update src/Umbraco.Web.Common/ModelsBuilder/NoopModelsBuilderDashboardProvider.cs

Co-authored-by: Ronald Barendse <ronald@barend.se>

Co-authored-by: Nikolaj Geisle <70372949+Zeegaan@users.noreply.github.com>
2022-07-01 08:48:05 +02:00
Nikolaj Geisle
f4e333c178 V10: fix build warnings infrastructure (#12369)
* Run code cleanup

* Run dotnet format

* Start manual fixes

* Manual fixing of warnings

* Fix nullability in columnalias

* Fix tests

* Fix up after merge

* Start updating after review

* Update editorconfig to contain new static & const rules

* Fix up editorconfig to not contain duplicate rules

* Fix up static member names

* Fix up according to review

* Update src/Umbraco.Infrastructure/DependencyInjection/UmbracoBuilder.DistributedCache.cs

Co-authored-by: Mole <nikolajlauridsen@protonmail.ch>

* Update src/Umbraco.Infrastructure/DependencyInjection/UmbracoBuilder.Repositories.cs

Co-authored-by: Mole <nikolajlauridsen@protonmail.ch>

* Update src/Umbraco.Infrastructure/DependencyInjection/UmbracoBuilder.Repositories.cs

Co-authored-by: Mole <nikolajlauridsen@protonmail.ch>

* Update src/Umbraco.Infrastructure/Examine/ContentIndexPopulator.cs

Co-authored-by: Mole <nikolajlauridsen@protonmail.ch>

* Update src/Umbraco.Infrastructure/Examine/ContentIndexPopulator.cs

Co-authored-by: Mole <nikolajlauridsen@protonmail.ch>

* Update src/Umbraco.Infrastructure/Examine/ContentValueSetValidator.cs

Co-authored-by: Mole <nikolajlauridsen@protonmail.ch>

* Update src/Umbraco.Infrastructure/Examine/ContentValueSetValidator.cs

Co-authored-by: Mole <nikolajlauridsen@protonmail.ch>

* Update src/Umbraco.Infrastructure/Examine/ContentValueSetValidator.cs

Co-authored-by: Mole <nikolajlauridsen@protonmail.ch>

* Update src/Umbraco.Infrastructure/Examine/ExamineUmbracoIndexingHandler.cs

Co-authored-by: Mole <nikolajlauridsen@protonmail.ch>

* Update src/Umbraco.Infrastructure/Examine/PublishedContentIndexPopulator.cs

Co-authored-by: Mole <nikolajlauridsen@protonmail.ch>

* Update src/Umbraco.Infrastructure/Extensions/InstanceIdentifiableExtensions.cs

Co-authored-by: Mole <nikolajlauridsen@protonmail.ch>

* Update src/Umbraco.Infrastructure/HostedServices/RecurringHostedServiceBase.cs

Co-authored-by: Mole <nikolajlauridsen@protonmail.ch>

* Update src/Umbraco.Infrastructure/HostedServices/ReportSiteTask.cs

Co-authored-by: Mole <nikolajlauridsen@protonmail.ch>

* Update src/Umbraco.Infrastructure/Logging/Serilog/LoggerConfigExtensions.cs

Co-authored-by: Mole <nikolajlauridsen@protonmail.ch>

* Update src/Umbraco.Infrastructure/Logging/Serilog/LoggerConfigExtensions.cs

Co-authored-by: Mole <nikolajlauridsen@protonmail.ch>

* Update src/Umbraco.Infrastructure/Macros/MacroTagParser.cs

Co-authored-by: Mole <nikolajlauridsen@protonmail.ch>

* Update src/Umbraco.Infrastructure/Macros/MacroTagParser.cs

Co-authored-by: Mole <nikolajlauridsen@protonmail.ch>

* Update src/Umbraco.Infrastructure/Macros/MacroTagParser.cs

Co-authored-by: Mole <nikolajlauridsen@protonmail.ch>

* Update src/Umbraco.Infrastructure/Migrations/Expressions/Alter/Table/IAlterTableColumnOptionBuilder.cs

Co-authored-by: Mole <nikolajlauridsen@protonmail.ch>

* Update src/Umbraco.Infrastructure/Migrations/Upgrade/V_10_0_0/AddMemberPropertiesAsColumns.cs

Co-authored-by: Mole <nikolajlauridsen@protonmail.ch>

* Update src/Umbraco.Infrastructure/ModelsBuilder/Building/TextBuilder.cs

Co-authored-by: Mole <nikolajlauridsen@protonmail.ch>

* Update src/Umbraco.Infrastructure/ModelsBuilder/Building/TextBuilder.cs

Co-authored-by: Mole <nikolajlauridsen@protonmail.ch>

* Update src/Umbraco.Infrastructure/ModelsBuilder/Building/TextBuilder.cs

Co-authored-by: Mole <nikolajlauridsen@protonmail.ch>

* Update src/Umbraco.Infrastructure/ModelsBuilder/Building/TextBuilder.cs

Co-authored-by: Mole <nikolajlauridsen@protonmail.ch>

* Update src/Umbraco.Infrastructure/ModelsBuilder/Building/TextBuilder.cs

Co-authored-by: Mole <nikolajlauridsen@protonmail.ch>

* Update src/Umbraco.Infrastructure/ModelsBuilder/Building/TextBuilder.cs

Co-authored-by: Mole <nikolajlauridsen@protonmail.ch>

* Update src/Umbraco.Infrastructure/ModelsBuilder/Building/TextBuilder.cs

Co-authored-by: Mole <nikolajlauridsen@protonmail.ch>

* Update src/Umbraco.Infrastructure/Persistence/Dtos/ExternalLoginDto.cs

Co-authored-by: Mole <nikolajlauridsen@protonmail.ch>

* Update src/Umbraco.Infrastructure/Persistence/Mappers/AccessMapper.cs

Co-authored-by: Mole <nikolajlauridsen@protonmail.ch>

* Update src/Umbraco.Infrastructure/Persistence/Mappers/AuditEntryMapper.cs

Co-authored-by: Mole <nikolajlauridsen@protonmail.ch>

* Update src/Umbraco.Infrastructure/Persistence/Mappers/MediaMapper.cs

Co-authored-by: Mole <nikolajlauridsen@protonmail.ch>

* Update src/Umbraco.Infrastructure/Persistence/Mappers/MemberMapper.cs

Co-authored-by: Mole <nikolajlauridsen@protonmail.ch>

* Update src/Umbraco.Infrastructure/Persistence/Mappers/PropertyGroupMapper.cs

Co-authored-by: Mole <nikolajlauridsen@protonmail.ch>

* Update src/Umbraco.Infrastructure/Persistence/Mappers/PropertyGroupMapper.cs

Co-authored-by: Mole <nikolajlauridsen@protonmail.ch>

* Update src/Umbraco.Infrastructure/Persistence/Mappers/PropertyTypeMapper.cs

Co-authored-by: Mole <nikolajlauridsen@protonmail.ch>

* Update src/Umbraco.Infrastructure/Persistence/Mappers/PropertyTypeMapper.cs

Co-authored-by: Mole <nikolajlauridsen@protonmail.ch>

* Update src/Umbraco.Infrastructure/Persistence/Mappers/RelationTypeMapper.cs

Co-authored-by: Mole <nikolajlauridsen@protonmail.ch>

* Update src/Umbraco.Infrastructure/Persistence/Mappers/RelationTypeMapper.cs

Co-authored-by: Mole <nikolajlauridsen@protonmail.ch>

* Update src/Umbraco.Infrastructure/Persistence/NPocoMapperCollectionBuilder.cs

Co-authored-by: Mole <nikolajlauridsen@protonmail.ch>

* Update src/Umbraco.Infrastructure/Persistence/Querying/ExpressionVisitorBase.cs

Co-authored-by: Mole <nikolajlauridsen@protonmail.ch>

* Update src/Umbraco.Infrastructure/Persistence/Repositories/Implement/ExternalLoginRepository.cs

Co-authored-by: Mole <nikolajlauridsen@protonmail.ch>

* Fix [..] to substring

* Fix after merge with 10/dev

* Fox ContentValueSetValidator.cs

* Update LoggerConfigExtensions

Co-authored-by: Nikolaj Geisle <niko737@edu.ucl.dk>
Co-authored-by: Mole <nikolajlauridsen@protonmail.ch>
2022-06-02 08:18:31 +02:00
Ronald Barendse
8e6e262c7f v10: Use ForceCreateDatabase during unattended install and extend GetUmbracoConnectionString extension methods (#12397)
* Add extension methods to get the Umbraco connection string/provider name from configuration

* Added tests for configuration extension methods.

* Fix issue with InstallMissingDatabase and ForceCreateDatabase

* Fix comments

* Revert casing change in GenerateConnectionString

* Re-add AddOptions (without config binding) to fix test

* Update src/Umbraco.Core/Configuration/Models/ConnectionStrings.cs

Co-authored-by: Ronald Barendse <ronald@barend.se>

* Update src/Umbraco.Core/Configuration/Models/ConnectionStrings.cs

* Update src/Umbraco.Infrastructure/Runtime/RuntimeState.cs

* Whitespace and documentation updates

* Add DatabaseProviderMetadataExtensions

* Filter before ordering

* Replace DataDirectory placeholder when setting connection string

Co-authored-by: Andy Butland <abutland73@gmail.com>
Co-authored-by: Bjarke Berg <mail@bergmania.dk>
2022-05-17 12:59:01 +02:00
Nikolaj Geisle
214bcceffc Merge v10/dev into infrastructure 2022-03-16 14:39:28 +01:00
Paul Johnson
3961c4c233 v10 SQLite support + distributed locking abstractions (#11922)
* Created Persistence.SQLite project skeleton.

* SQLite database initialization

* Various changes and hacks to make things work.

* WIP integration tests

* Fix thread safety tests

* Fix tests that relied on tie breaker sorting.

Spent a fair amount of time looking for a less lazy fix but gave up.

* Convert right join to left join ContentTypeRepository.PerformGetByQuery
SQLite doesn't support right join

* Fix test Can_Generate_Delete_SubQuery_Statement

Worth noting that NPoco.DatabaseTypes.SQLiteDatabaseType doesn't override
EscapeSqlIdentifier so NPoco will escape with [].

SQLite docs say > "A keyword enclosed in square brackets is an identifier.
This is not standard SQL.
This quoting mechanism is used by MS Access and SQL Server and is
included in SQLite for compatibility."

Also could have updated SqliteSyntaxProvider to match npoco but
decided against it.

* Fixes for paginated custom order by

* Fix tests broken by lack of unique indexes.

* Fix SqlServerTableByTableTest tests.

These tests didn't actually do anything as the tables already exist so schema creator just returned.

Did however point out that the default implementation for DoesTableExist just returns false so added a default naive implementation.

* Fix ValidateLoginSession - SelectTop must come later

* dry up database cleanup

* Fix up db migration tests.
We can't drop pk in sqlite without recreating table.
Test looks to be testing that add column works as intended which we can test.

* Prevent schema creation errors.

* SQLite ignore lock tests, WAL back on.

* Fix package schema tests

* Fix NPocoFetchTests - case sensitivity not under test

* Fix AdvancedMigrationTests (where possible)

Migrations probably need a good look later.
Maybe nuke old migrations and only support moving to v10 from v9.

If we do that can do some cleanup.

* Cleanup test database configuration

* Run integration tests against SQLite on build agent.

* Drop MS.Data.SQLite
System.Data.SQLite was quicker to roll out due to more CLR type mapping

* YAML

* Skip Umbraco.Tests.Integration.SqlCe

* Drop SqlServerTableByTable tests.

Until this week they did nothing anyway as they with NewSchemaPerTest
so the tests all passed as CreateTable was no op (already exists).

Also all of the tables are created in an empty database by SchemaValidationTest.cs
DatabaseSchemaCreation_Produces_DatabaseSchemaResult_With_Zero_Errors

* Might aswell run against macOS also.

* Copy azure pipelines task header layout

* Delete SQLCe projects

* Remove SQL CE specific code.

* Remove SQL CE NuSpec, template params, build script setup

* Delete umbraco-netcore-only.sln

* Add SkipTests solution configuration and use for codeql

* Remove reference to deleted nuspec file.

* Refactor ConnectionStrings WRT DataDirectory placeholder & ProviderName.

At this point you can try out SQLite support by setting the following
in appsettings.json and then completing the install process.

"ConnectionStrings": {
  "umbracoDbDSN": "Data Source=|DataDirectory|/umbraco.sqlite",
  "umbracoDbDSN_ProviderName": "System.Data.SQLite"
},

Not currently possible via installer UI without provider name pre-set in
configuration.

* Switch to Microsoft.Data.Sqlite

Some gross hacks but will be good to find out if this works
with apple silicon.

* Enable selection of SQLite via installer UI (also quick install)

* Remove SqlServerDbProviderFactoryCreator to cleanup a TODO

* Move SQL Server support to its own class library

* Add persistence dependencies to Umbraco.CMS metapackage

* Bugfix packages delete query

Created invalid query for SQLite.

* Try out cypress tests Linux + SQLite

* Prevent cypress test artifact upload failure on attempt 2+

* LocalDb bugfixes

* Drop redundant enum

* Move SqlClient constant

* Misc whitespace

* Remove IsSqlCe extension (TODO: drop non 9->10 migrations later).

* Umbraco.Persistence.* -> Umbraco.Cms.Persistence.*

* Display quick install defaults and per provider default database name.

* Misc remove old comment

* little re-arrange

* Remove almost all usages of IsSqlite extension.

* visual adjustments

* Custom Database Configuration is last step and should then say Install.

* use text instead of disabled inputs

* move legend, rename to Install

* Update SqlMainDomLock to work without distributed locks.

* Added IDistributedLockingMechanism interface and in memory impl.

* Drop locking from ISqlSyntaxProvider & wire up scope to abstraction.

* Added SqlServerDistributedLockingMechanism

* Move distributed locking interfaces and exceptions to Core + xmldocs.

* Fix tests, Misc cleanup, Add SQL distributed locking integration tests

* Provide mechanism to specify DistributedLockingMechanism in config
(even if added by composer)

* Nomplementation -> NoImplementation

* Fix misleading comment

* Integration tests use SqlServerDistributedLockingMechanism when possible

* Handle up-gradable locks SqlServerDistributedLockingMechanism.
TODO: InMemoryDistributedLockingMechanism.

Note: Nuked SqlServerDistributedLockingMechanismTests, will still sleep
at night.
Is covered by Umbraco.Cms.Tests.Integration.Umbraco.Infrastructure.Persistence.LockTests

* Make tests pass for InMemoryDistributedLockingMechanism, pretty hacky.

* Tweak constraints on WithCollectionBuilder so i can drop bad constructor

* Added SqliteDistributedLockingMechanism

* Dropped InMemoryDistributedMechanism + magic

InMemoryDistributedMechanism was pretty rubbish and now we have
a decent implementation for SQLite as we no longer block readers
see 8d1f42b.

Also drop the CollectionBuilder setup, instead do the same as we do
for syntax providers etc, it's more automagical so we never require an
explicit selection although we are allowing for it.

However keeping the optional IUmbracoBuilder constructor param for
CollectionBuilders as it's extremely useful.

* Fix quick install "" database name.

* Hide Database Configuration section when a connection string is pre-set.

Doesn't seem worth it to extract db name from connection string.

* Ensure wal test 2+

* Fix logging inconsistencies.

* Ensure in transaction when obtaining locks + no-op the SQLite read lock.

There's no point in running the query just to make a single test pass.

* Fix installer database display names

* Allow SQLite shared cache without losing deferred transactions

* Opt into shared cache for new SQLite databases + fix filename

* Fix misc inconsistency in .gitignore

* Prefer our interceptor interface

* Restore DEBUG_DATABASES code OnConnectionOpened in case it's used.

* Back to private cache.

* Added retry strategy for SQLite + refactor out SQL server specific stuff

* Fix SQL server tests.

* Misc - Orphaned comment, incorrect casing.

* InMemory SQLite test database & turn shared cache back on everywhere.

Co-authored-by: Niels Lyngsø <niels.lyngso@gmail.com>
2022-03-11 17:14:20 +01:00
Nikolaj Geisle
ed13237177 Fixing more errors with nullable reference types 2022-02-24 09:24:56 +01:00
Nikolaj Geisle
a8cf6ee127 More work on nullable references 2022-02-22 13:35:32 +01:00
Bjarke Berg
7971f36b78 Add check for PluginControllerAttribute and compare area name (#11911)
* Add check for PluginControllerAttribute and compare area name

* Added null check
2022-01-27 10:32:56 +01:00
Nikolaj Geisle
bc6d8b5ece v9: Throw error on duplicate routes (#11774)
* Add conflicting route service and check

* Added testclass

* Cleanup

* Update src/Umbraco.Web.BackOffice/Services/ConflictingRouteService.cs

Co-authored-by: Elitsa Marinovska <21998037+elit0451@users.noreply.github.com>

* Implemented out variable

Co-authored-by: Elitsa Marinovska <elm@umbraco.dk>
Co-authored-by: Elitsa Marinovska <21998037+elit0451@users.noreply.github.com>
2021-12-22 13:03:38 +01:00
Ronald Barendse
ab28251558 Use RuntimeState to determine whether to install missing database 2021-09-17 16:30:46 +02:00
Bjarke Berg
9e4dead0f5 Merge remote-tracking branch 'origin/v9/dev' into v9/task/package-refactor
# Conflicts:
#	src/Umbraco.Core/Migrations/IMigration.cs
#	src/Umbraco.Core/Migrations/MigrationPlan.cs
#	src/Umbraco.Infrastructure/Migrations/IMigrationContext.cs
#	src/Umbraco.Infrastructure/Runtime/RuntimeState.cs
#	src/Umbraco.Tests.UnitTests/Umbraco.Infrastructure/Migrations/MigrationTests.cs
#	src/Umbraco.Web.UI.NetCore/umbraco/config/lang/en.xml
#	src/Umbraco.Web.UI.NetCore/umbraco/config/lang/en_us.xml
2021-06-29 14:23:08 +02:00
Shannon
b1c6b02dd7 Merge remote-tracking branch 'origin/v9/task/package-refactor' into v9/task/package-migrations-backoffice
# Conflicts:
#	src/Umbraco.Core/Extensions/XmlExtensions.cs
#	src/Umbraco.Core/Packaging/PackageMigrationPlan.cs
#	src/Umbraco.Infrastructure/Runtime/RuntimeState.cs
2021-06-18 12:48:44 -06:00
Shannon Deminick
d7c0c6a932 Adjust the runtime state and keep disabling unattended package migrations simple (#10486)
* Clean up and changes to backoffice for the nuget only packages

* temp commit of package logic removal

* Lots of package code cleanup and removal

* Removes old package data from the test package xml

* Updates packaging code to take in XDocument instead of a file since we'll not be dealing with files, starts creating expressions for the package migrations scripting.

* fixing tests

* Fixes runtime state and boot failed middleware so that it actually runs. Separates out unattended install/upgrade into notification handlers.

* Gets unattended package migrations working and running

* Gets embedded package.xml resources able to install from package migration.

* Implements automatic package migrations for package that just declare an xml data manifest.

* fix build

* small cleanups

* fix build

* adds some tests

* Fix export test

* Fix newlines in test for linux

* Typo

* removes old todos and updates AutomaticPackgeMigrationPlan to use getter with backing field.

* Update dictionary package data to use GUID

* Ensures macros are packaged and used with their GUID

* Ensures the GUID for doc types and media types remains consistent for package installation based on what is in the xml.

* fix automatic migrations to not validate initial state, fixes packaging GUIDs for multiple entities.

* Added guids to embedded test packages (Some tests are still failing)

* Fix one more test

* Fixes up Key vs Id, moves tests to correct namespace, fix tests

* Fixes Dictionary packaging to ensure an xml hierarchy

* Fixes tests

* fixes package xml

* Removes the runtime PackageMigrations state, the state is just run if unattended migrations are disabled.

* change log level

* Small clean up and reuse of attribute

Co-authored-by: Bjarke Berg <mail@bergmania.dk>
2021-06-17 10:00:49 +02:00
Shannon
ad84c1591e Starts implementing ability to manually run pending migrations from the back office 2021-06-16 15:34:20 -06:00
Shannon
7ec01f232f Removes the runtime PackageMigrations state, the state is just run if unattended migrations are disabled. 2021-06-16 13:53:01 -06:00
Shannon Deminick
de2fb47b78 Implements unattended package migrations for both explicit and implicit migrations (#10418)
* Clean up and changes to backoffice for the nuget only packages

* temp commit of package logic removal

* Lots of package code cleanup and removal

* Removes old package data from the test package xml

* Updates packaging code to take in XDocument instead of a file since we'll not be dealing with files, starts creating expressions for the package migrations scripting.

* fixing tests

* Fixes runtime state and boot failed middleware so that it actually runs. Separates out unattended install/upgrade into notification handlers.

* Gets unattended package migrations working and running

* Gets embedded package.xml resources able to install from package migration.

* Implements automatic package migrations for package that just declare an xml data manifest.

* fix build

* small cleanups

* fix build

* adds some tests

* Fix export test

* Fix newlines in test for linux

* Typo

* removes old todos and updates AutomaticPackgeMigrationPlan to use getter with backing field.

Co-authored-by: Bjarke Berg <mail@bergmania.dk>
2021-06-15 07:05:57 +02:00
Shannon
10016c74f9 removes old todos and updates AutomaticPackgeMigrationPlan to use getter with backing field. 2021-06-14 14:22:11 -06:00
Shannon
daceca54a3 Implements automatic package migrations for package that just declare an xml data manifest. 2021-06-11 10:50:35 -06:00
Shannon
2114667e65 Gets unattended package migrations working and running 2021-06-10 10:21:58 -06:00
Shannon
144014dc73 Fixes runtime state and boot failed middleware so that it actually runs. Separates out unattended install/upgrade into notification handlers. 2021-06-10 08:06:17 -06:00