Commit Graph

21 Commits

Author SHA1 Message Date
Nikolaj Geisle
4ebd20bb66 Merge remote-tracking branch 'origin/v10/dev' into v10/feature/nullable-reference-types-in-Umbraco-Core
# Conflicts:
#	build/build.ps1
#	src/Umbraco.Core/Configuration/ConfigConnectionString.cs
#	src/Umbraco.Core/Configuration/Models/ConnectionStrings.cs
#	src/Umbraco.Core/Install/InstallSteps/TelemetryIdentifierStep.cs
#	src/Umbraco.Core/Models/ContentType.cs
#	src/Umbraco.Infrastructure/Migrations/Install/DatabaseBuilder.cs
#	tests/Umbraco.Tests.AcceptanceTest/package.json
2022-03-16 13:00:38 +01:00
Paul Johnson
3961c4c233 v10 SQLite support + distributed locking abstractions (#11922)
* Created Persistence.SQLite project skeleton.

* SQLite database initialization

* Various changes and hacks to make things work.

* WIP integration tests

* Fix thread safety tests

* Fix tests that relied on tie breaker sorting.

Spent a fair amount of time looking for a less lazy fix but gave up.

* Convert right join to left join ContentTypeRepository.PerformGetByQuery
SQLite doesn't support right join

* Fix test Can_Generate_Delete_SubQuery_Statement

Worth noting that NPoco.DatabaseTypes.SQLiteDatabaseType doesn't override
EscapeSqlIdentifier so NPoco will escape with [].

SQLite docs say > "A keyword enclosed in square brackets is an identifier.
This is not standard SQL.
This quoting mechanism is used by MS Access and SQL Server and is
included in SQLite for compatibility."

Also could have updated SqliteSyntaxProvider to match npoco but
decided against it.

* Fixes for paginated custom order by

* Fix tests broken by lack of unique indexes.

* Fix SqlServerTableByTableTest tests.

These tests didn't actually do anything as the tables already exist so schema creator just returned.

Did however point out that the default implementation for DoesTableExist just returns false so added a default naive implementation.

* Fix ValidateLoginSession - SelectTop must come later

* dry up database cleanup

* Fix up db migration tests.
We can't drop pk in sqlite without recreating table.
Test looks to be testing that add column works as intended which we can test.

* Prevent schema creation errors.

* SQLite ignore lock tests, WAL back on.

* Fix package schema tests

* Fix NPocoFetchTests - case sensitivity not under test

* Fix AdvancedMigrationTests (where possible)

Migrations probably need a good look later.
Maybe nuke old migrations and only support moving to v10 from v9.

If we do that can do some cleanup.

* Cleanup test database configuration

* Run integration tests against SQLite on build agent.

* Drop MS.Data.SQLite
System.Data.SQLite was quicker to roll out due to more CLR type mapping

* YAML

* Skip Umbraco.Tests.Integration.SqlCe

* Drop SqlServerTableByTable tests.

Until this week they did nothing anyway as they with NewSchemaPerTest
so the tests all passed as CreateTable was no op (already exists).

Also all of the tables are created in an empty database by SchemaValidationTest.cs
DatabaseSchemaCreation_Produces_DatabaseSchemaResult_With_Zero_Errors

* Might aswell run against macOS also.

* Copy azure pipelines task header layout

* Delete SQLCe projects

* Remove SQL CE specific code.

* Remove SQL CE NuSpec, template params, build script setup

* Delete umbraco-netcore-only.sln

* Add SkipTests solution configuration and use for codeql

* Remove reference to deleted nuspec file.

* Refactor ConnectionStrings WRT DataDirectory placeholder & ProviderName.

At this point you can try out SQLite support by setting the following
in appsettings.json and then completing the install process.

"ConnectionStrings": {
  "umbracoDbDSN": "Data Source=|DataDirectory|/umbraco.sqlite",
  "umbracoDbDSN_ProviderName": "System.Data.SQLite"
},

Not currently possible via installer UI without provider name pre-set in
configuration.

* Switch to Microsoft.Data.Sqlite

Some gross hacks but will be good to find out if this works
with apple silicon.

* Enable selection of SQLite via installer UI (also quick install)

* Remove SqlServerDbProviderFactoryCreator to cleanup a TODO

* Move SQL Server support to its own class library

* Add persistence dependencies to Umbraco.CMS metapackage

* Bugfix packages delete query

Created invalid query for SQLite.

* Try out cypress tests Linux + SQLite

* Prevent cypress test artifact upload failure on attempt 2+

* LocalDb bugfixes

* Drop redundant enum

* Move SqlClient constant

* Misc whitespace

* Remove IsSqlCe extension (TODO: drop non 9->10 migrations later).

* Umbraco.Persistence.* -> Umbraco.Cms.Persistence.*

* Display quick install defaults and per provider default database name.

* Misc remove old comment

* little re-arrange

* Remove almost all usages of IsSqlite extension.

* visual adjustments

* Custom Database Configuration is last step and should then say Install.

* use text instead of disabled inputs

* move legend, rename to Install

* Update SqlMainDomLock to work without distributed locks.

* Added IDistributedLockingMechanism interface and in memory impl.

* Drop locking from ISqlSyntaxProvider & wire up scope to abstraction.

* Added SqlServerDistributedLockingMechanism

* Move distributed locking interfaces and exceptions to Core + xmldocs.

* Fix tests, Misc cleanup, Add SQL distributed locking integration tests

* Provide mechanism to specify DistributedLockingMechanism in config
(even if added by composer)

* Nomplementation -> NoImplementation

* Fix misleading comment

* Integration tests use SqlServerDistributedLockingMechanism when possible

* Handle up-gradable locks SqlServerDistributedLockingMechanism.
TODO: InMemoryDistributedLockingMechanism.

Note: Nuked SqlServerDistributedLockingMechanismTests, will still sleep
at night.
Is covered by Umbraco.Cms.Tests.Integration.Umbraco.Infrastructure.Persistence.LockTests

* Make tests pass for InMemoryDistributedLockingMechanism, pretty hacky.

* Tweak constraints on WithCollectionBuilder so i can drop bad constructor

* Added SqliteDistributedLockingMechanism

* Dropped InMemoryDistributedMechanism + magic

InMemoryDistributedMechanism was pretty rubbish and now we have
a decent implementation for SQLite as we no longer block readers
see 8d1f42b.

Also drop the CollectionBuilder setup, instead do the same as we do
for syntax providers etc, it's more automagical so we never require an
explicit selection although we are allowing for it.

However keeping the optional IUmbracoBuilder constructor param for
CollectionBuilders as it's extremely useful.

* Fix quick install "" database name.

* Hide Database Configuration section when a connection string is pre-set.

Doesn't seem worth it to extract db name from connection string.

* Ensure wal test 2+

* Fix logging inconsistencies.

* Ensure in transaction when obtaining locks + no-op the SQLite read lock.

There's no point in running the query just to make a single test pass.

* Fix installer database display names

* Allow SQLite shared cache without losing deferred transactions

* Opt into shared cache for new SQLite databases + fix filename

* Fix misc inconsistency in .gitignore

* Prefer our interceptor interface

* Restore DEBUG_DATABASES code OnConnectionOpened in case it's used.

* Back to private cache.

* Added retry strategy for SQLite + refactor out SQL server specific stuff

* Fix SQL server tests.

* Misc - Orphaned comment, incorrect casing.

* InMemory SQLite test database & turn shared cache back on everywhere.

Co-authored-by: Niels Lyngsø <niels.lyngso@gmail.com>
2022-03-11 17:14:20 +01:00
Nikolaj Geisle
945b115067 Fix more nullability errors 2022-02-15 14:41:06 +01:00
Mole
3261a6f71d Fix up for V9 2022-01-26 12:12:59 +01:00
Mole
2ddb3e13e5 Fix breaking changes 2022-01-26 10:05:01 +01:00
Sebastiaan Janssen
8eb3ef65f6 Merge branch 'v9/dev' into v9/contrib 2022-01-21 10:02:37 +01:00
Elitsa Marinovska
229ca989eb V9: Allowlisting remote URLs for displaying content on the content dashboard (#11825)
* Fixing ContentDashboardSettings to work when set in the config

* Moving file in Models folder and adding ContentDashboardUrlAllowlist setting

* Implementing allowlist for content dashboard base url

* Cleanup

* Error msg vs log msg
2022-01-18 15:37:31 +01:00
Bjarke Berg
642c216f94 Serve Media and App_Plugins using WebRootFileProvider (and allow changing the physical media path) (#11783)
* Allow changing UmbracoMediaPath to an absolute path. Also ensure Imagesharp are handing requests outside of the wwwroot folder.

* Let UmbracoMediaUrl fallback to UmbracoMediaPath when empty

* Add FileSystemFileProvider to expose an IFileSystem as IFileProvider

* Replace IUmbracoMediaFileProvider with IFileProviderFactory implementation

* Fix issue resolving relative paths when media URL has changed

* Remove FileSystemFileProvider and require explicitly implementing IFileProviderFactory

* Update tests (UnauthorizedAccessException isn't thrown anymore for rooted files)

* Update test to use UmbracoMediaUrl

* Add UmbracoMediaPhysicalRootPath global setting

* Remove MediaFileManagerImageProvider and use composited file providers

* Move CreateFileProvider to IFileSystem extension method

* Add rooted path tests

Co-authored-by: Ronald Barendse <ronald@barend.se>
2022-01-06 13:35:24 +01:00
Mole
04d20269ab Merge CharCollection and UserDefinedCharCollection
Otherwise only CharCollection would work, and that's obsolete, now you can use either, but UserDefinedCharCollection takes priority
2021-12-22 13:04:40 +01:00
Mole
53e5a25df8 Remove ugly CharCollection hack
By adding a new configuration and mapping the old CharCollection to that, we can get around having to return null the first time, by obsoleting the old one and redirecting to the new GetCharReplacements method
2021-12-22 11:29:04 +01:00
Bjarke Berg
915f1cb34c V9: Fix for migration of non-default configurated users/members (#11684)
* https://github.com/umbraco/Umbraco-CMS/issues/11366
Fallback to try login using super legacy HMACSHA1 even when the algorithm is stated as being HMACSHA256. The issue is that v8 saves HMACSHA256 on the user, but when configured to use legacy encoding it actually uses HMACSHA1

* Support migration of members with:
UseLegacyEncoding+Clear
UseLegacyEncoding+Encrypted (Requires machine key)
UseLegacyEncoding+Hashed

* Fixes unit tests

* Avoid exceptions + unit tests

* Save unknown algorithm if we dont know it, instead of persisting a wrong algorithm.

* Added setting to enable clear text password rehashes.

* Removed support for migration of clear text passwords

* Fixed unit test
2021-12-02 10:54:24 +00:00
Bjarke Berg
2a29cdc6c6 Allow opt out of import embedded schema file (#11296)
* Introduce an opt-out options from the import of embedded schema files.

* Moved the initialization of the static service provider into CoreRuntime as this runs before the IStartupFilters, and otherwise the static service provider is not available in hosted services. E.g. for migrations

* fix build

* Minor code tidy and naming alignment.

* Update src/Umbraco.Web.UI/Umbraco.Web.UI.csproj

* Removed default installation of starter kit.

Co-authored-by: Andy Butland <abutland73@gmail.com>
2021-10-06 11:13:59 +02:00
Bjarke Berg
4f2cb09939 Added POC of basic auth middleware 2021-08-05 21:42:47 +02:00
Bjarke Berg
74e2b7aca2 Refactored the initialization of settings and started a json schema project 2021-06-24 08:47:37 +02:00
Bjarke Berg
fbcaa6adea https://github.com/umbraco/Umbraco-CMS/issues/10060 - Fixed issue where appsettings section for modelsbuilder was not used. 2021-03-25 16:17:46 +01:00
Shannon
eba659b9ee Gets in-memory runtime minifiy cache working, ensures no files can be accessed at the root, adds config to enable in-memory and configure a cache buster including a timestamp cache buster. 2021-03-17 17:44:52 +11:00
Bjarke Berg
2b89839724 Netcore: Get rid of default files in config folder (#9966)
* TourController now uses the core tours from embedded resources

* Moved tinyMceConfig.*.config to IOptions

* Embedded the default grid.editors.config.js

* Fixed issue when saving grid with an empty media cell

* Logviewer now uses sql as database instead of file.

* Remove config folder from build script and nuget pacakges.

* Removing auto-generated Id added to appsettings.json

* Update src/Umbraco.Web.BackOffice/Controllers/TourController.cs

Co-authored-by: Elitsa Marinovska <elm@umbraco.dk>
Co-authored-by: Elitsa Marinovska <21998037+elit0451@users.noreply.github.com>
2021-03-11 13:20:46 +01:00
Bjarke Berg
24d4b4c9fb Added validation and moved Unattended Settings into their own model. + Added DataAnnonation validation on all our settings 2021-03-07 19:20:16 +01:00
Mole
bf41c2eeaa Netcore: Align namespaces (#9801)
* Rename Umbraco.Core namespace to Umbraco.Cms.Core

* Move extension methods in core project to Umbraco.Extensions

* Move extension methods in core project to Umbraco.Extensions

* Rename Umbraco.Examine namespace to Umbraco.Cms.Examine

* Move examine extensions to Umbraco.Extensions namespace

* Reflect changed namespaces in Builder and fix unit tests

* Adjust namespace in Umbraco.ModelsBuilder.Embedded

* Adjust namespace in Umbraco.Persistence.SqlCe

* Adjust namespace in Umbraco.PublishedCache.NuCache

* Align namespaces in Umbraco.Web.BackOffice

* Align namespaces in Umbraco.Web.Common

* Ensure that SqlCeSupport is still enabled after changing the namespace

* Align namespaces in Umbraco.Web.Website

* Align namespaces in Umbraco.Web.UI.NetCore

* Align namespaces in Umbraco.Tests.Common

* Align namespaces in Umbraco.Tests.UnitTests

* Align namespaces in Umbraco.Tests.Integration

* Fix errors caused by changed namespaces

* Fix integration tests

* Undo the Umbraco.Examine.Lucene namespace change

This breaks integration tests on linux, since the namespace wont exists there because it's only used on windows.

* Fix merge

* Fix Merge
2021-02-18 11:06:02 +01:00
Shannon
2848f4083b notes - why isn't this building? 2021-01-04 12:01:52 +11:00
Shannon
e785ac28a3 Moves more services and registrations "up", removes AspNetCoreComposer logic and moves to AddWebComponents 2020-12-24 14:29:26 +11:00