Commit Graph

190 Commits

Author SHA1 Message Date
Nikolaj Geisle
4f3d680f06 V10: Build warnings in Web.Website (#12332)
* add new rule to globalconfig

* Fix warnings in Web.Website

* Fix more warnings in Web.Website

* Fix more build warnings in Web.Website

* Fix more warnings in Web.Website

* Fix tests

* Fix proper constructor call

* Fix not being able to run project

* Fix Obsolete method

Co-authored-by: Nikolaj Geisle <niko737@edu.ucl.dk>
2022-05-06 15:06:39 +02:00
Bjarke Berg
5636d4baf7 Fixed namespace that was original an incorrect merge from v8 to v9 (#12347) 2022-05-04 08:10:27 +02:00
Bjarke Berg
84c1799049 Merge remote-tracking branch 'origin/v10/contrib' into v10/dev 2022-05-03 21:36:30 +02:00
Chad
0561d4b5a1 Async support for content finders. Improve log performance. (#12340) 2022-05-03 21:29:15 +02:00
Bjarke Berg
13f6d4791c Move umbraco views to static assets and make that an RCL + Embedded language files (#12324)
* RCL for static assets to replace the nuspec

* Fix build

* Fix unit tests

* clean up in build.ps1

* Removed test (lang files will be removed later anyway)

* Fixed namespaces.. + Ensure we set web root path if missing (e.g. wwwroot folder do not exist) + Added StaticWebAssetBasePath

* fixed namespace

* cleanup

* Set root variable

* Added static assets

* Experimenting with StaticWebAssetBasePath

* Embedded lang files into Umbraco.Core

* Removed legacy test. New test can be implemented instead

* Fixed tests

* clean up

* Fix merge issue
2022-05-02 19:38:33 +02:00
Ronald Barendse
f00bfc408e v10: Make language name editable (#12243)
* Update language models to get and set manual name

* Save custom language name in controller

* Rewrite AngularJS language edit view and controller

* Cleanup language overview

* Remove icon from language overview

* Make styling of control group the same as properties

* Ensure both ISO code and culture name are set in language model

* Use new language model constructor

* Update tests to use new language constructor

* Update culture name in dictionary package export

* Use language name in dictionary

* Fix language nullability issues

* Cleanup GetAllCultures and added null checks

* Re-add obsolete constructors

* Make language name required and update Cypress test

* Fix routing/saveNewLanguages Cypress test

* Make language name optional (improved backwards compatibility)

Co-authored-by: Ronald Barendse <ronald@panoramastudios.nl>
2022-05-02 15:42:19 +02:00
Bjarke Berg
11f9bcbea2 Update examine to version 3 (#12307)
* Updating to Examine 3 beta. Currently not building due to missing support for updating values in the OnTransformingIndexValues event

* Updated to beta 6  - It builds, but we need to fix tests

* Fixed issue with values being passed into nested lists.

* Fixed issue in test

* Updated to examine beta.9

* Fixed breaking changes
2022-05-02 14:11:46 +02:00
Jacob Overgaard
140029c38c v10: npm updates for april (#12288)
* Upgraded MailKit

* Upgraded Microsoft.* packages

* Upgrades various nuget packages to latest version

* Upgrades various nuget packages related to test

* update package-lock.json to new npm format

* bump all angular dependencies to latest 1.8.3

* bump jquery to 3.6.0

* remove dubious 'npm' dependency and set engines versions to avoid warnings

* bump signalr dependency from v3 to v6

* bump 'underscore' from 1.12.1 to 1.13.1

* bump 'diff' from 3.5.0 to 5.0.0

* bump 'clipboard' from 2.0.4 to 2.0.10

* bump 'flatpickr' from 4.6.9 to 4.6.13

* bump 'moment' from 2.22.2 to 2.29.3

* bump 'spectrum-colorpicker2' from 2.0.8 to 2.0.9

* bump 'underscore' from 1.13.1 to 1.13.2

* update 'less' 'autoprefixer' 'cssnano' and accompanying gulp tasks to latest

* bump all gulp dependencies

except gulp-imagemin since we do not yet support esmodules in the backoffice client

* remove 'fs' since the package is marked as malicious and do not seem to be used

* update AcceptanceTests lockfile to latest npm + dependency updates

* bump '@babel/core' from 7.17.5 to 7.17.9

* bump karma dependencies to latest

* bump 'animejs' from 2.2.0 to 3.2.1

* bump 'jasmine-core' from 3.5.0 to 4.1.0

* remove deduped 'caniuse-lite'

* bump wicg-inert from ^3.0.2 to 3.1.1

* bump 'nouislider' from 15.5.0 to 15.5.1

* add slight delay for confirmation overlay

* ignore postinstall task for acceptance tests since env file is generated manually

* upgrade cypress to 9.5.4

* add wait step to content.ts tests

* Revert "ignore postinstall task for acceptance tests since env file is generated manually"

This reverts commit 61acebfda2502499480f31d8450c889d4c5c6f11.

* increate wait amount of content rollback tests

* Revert "upgrade cypress to 9.5.4"

This reverts commit 2febf139a41938ae005754abd958fdec1f598991.

* Revert "update AcceptanceTests lockfile to latest npm + dependency updates"

This reverts commit b45d20b3cdcae6b17d4024c7085c5e6a3e3e2a83.

* update lockfile version

* update umbraco-cypress-testhelpers to latest

Co-authored-by: Bjarke Berg <mail@bergmania.dk>
2022-05-02 12:21:41 +02:00
Bjarke Berg
285c7e75a2 https://github.com/umbraco/Umbraco-CMS/issues/12276 (#12295)
Skip surrogate pairs (e.g. emojis) when making clean stings.
2022-04-29 13:46:19 +02:00
Ronald Barendse
1a82e0854a v10: Update to ImageSharp v2 (#12185)
* Update to ImageSharp 2.1.0 and ImageSharp.Web 2.0.0-alpha.0.23

* Rename CachedNameLength to CacheHashLength and add CacheFolderDepth setting

* Replace PhysicalFileSystemProvider with WebRootImageProvider

* Support EXIF-orientation in image dimention extractor

* Remove virtual methods on FileProviderImageProvider

* Simplify FileInfoImageResolver

* Update to SixLabors.ImageSharp.Web 2.0.0-alpha.0.25 and remove custom providers

* Make CropWebProcessor EXIF orientation-aware

* Improve width/height sanitization

* Also use 'v' as cache buster value

* Add WebP to supported image file types

* Update to SixLabors.ImageSharp.Web 2.0.0-alpha.0.27 and fix test

* Fix rounding error and add test cases

* Update to newest and stable releases

* Move ImageSharpImageUrlGenerator to Umbraco.Web.Common

* Use IConfigureOptions to configure ImageSharp options

* Implement IEquatable on ImageUrlGenerationOptions classes

* Fix empty/null values in image URL generation and corresponding tests

* Use IsSupportedImageFormat extension method

* Remove unneeded reflection
2022-04-29 13:16:24 +02:00
Nikolaj Geisle
cf2b9a0f21 V10: Move core services to core project (#12314)
* Move AuditService to core project

* Move two factor login service to core

* Move ServerRegistrationService to core

* Move BasicAuthService to Core project

* Move IdKeyMap to core project

* Added CacheInstructionService to the infrastructure namespace

* Move DataTypeService to core namespace

* Update CacheInstructionService.cs to use CoreScopeProvider

* Move core editors to core

* Move more Property editors and configuration

* Remove obsoleted constructors in internal classes

* Update PropertyEditors to use new ctors

* Fix propertyEditors to use new ctors

* Use the right property editor constructors

* add DI in the property method

* Update grid to use new ctor

* Fix non-assignment of variable

* Apply suggestions from code review

Co-authored-by: Mole <nikolajlauridsen@protonmail.ch>

* Fix suggestions from code review

Co-authored-by: Nikolaj Geisle <niko737@edu.ucl.dk>
Co-authored-by: Kevin Jump <kevin@thejumps.co.uk>
Co-authored-by: Mole <nikolajlauridsen@protonmail.ch>
2022-04-29 11:52:58 +02:00
Paul Johnson
95aa143db0 v10 make migration from v9 less painful WRT IScope (#12293)
* Restore IEventDispatcher

* Fix breaking changes WRT IScopeProvider and IScope

* Update internal usage.

* Update src/Umbraco.Core/Services/UserService.cs

* Better obsolete message

Co-authored-by: Mole <nikolajlauridsen@protonmail.ch>
2022-04-26 10:22:37 +01:00
Paul Johnson
551f523079 V10 - Reduce usage of temp AspNetCoreHostingEnvironment (#12270)
* Move MapPathWebRoot & MapPathContentRoot to extension methods.

* Set AspNetCoreHostingEnvironment ApplicationId without service provider.

* Drop some usages of TempHostingEnvironment

* Logging setup cleanup - AppDomainId has no value (it is always "1")

Creating new AppDomains isn't possible in .Net core.
Attempting to do so results in a platform exception.

A dotnet core process has a single AppDomain instance whose Id property
is always the integer 1.

* Setup logging without IHostingEnvironment

* Mark IUmbracoBuilder.BuilderHostingEnvironment obsolete
And remove internal usages.

* Typeloader no longer uses umbraco-types.list

* Added UmbracoHost - setup serilog two-stage init.

* Add ApplicationIdEnricher

* Defensive tweaks for those not using UmbracoHost

* Drop UmbracoHost static class

* Setup runtime logger without host builder extensions.

* Prevent RefreshingRazorViewEngine related explosions

* Filescoped namespaces for new classes.

* Apply suggestions from code review
2022-04-25 16:27:53 +02:00
Paul Johnson
2df6c937fd Fix flakey write lock test. (#12294)
And reduce logging noise.
2022-04-25 09:33:03 +02:00
Bjarke Berg
c441a26dc7 Merge remote-tracking branch 'origin/v9/dev' into v10/dev 2022-04-22 14:28:56 +02:00
Nikolaj Geisle
70a75d8c9b Merge remote-tracking branch 'origin/v10/dev' into v10/feature/nullable-reference-types-in-Umbraco.Web.Backoffice
# Conflicts:
#	src/Umbraco.Core/Models/Mapping/MemberTabsAndPropertiesMapper.cs
#	src/Umbraco.Core/Models/Member.cs
#	src/Umbraco.Core/Models/Membership/User.cs
#	src/Umbraco.Core/Services/MemberService.cs
#	src/Umbraco.Infrastructure/Security/BackOfficeUserStore.cs
#	src/Umbraco.Infrastructure/Security/MemberUserStore.cs
2022-04-22 12:32:57 +02:00
Nikolaj Geisle
fd8750f99a Try fixing flaky language test (#12281) 2022-04-22 09:26:12 +02:00
Bjarke Berg
379d2e3e4e Dependency updates (#12283)
* Upgraded MailKit

* Upgraded Microsoft.* packages

* Upgrades various nuget packages to latest version

* Upgrades various nuget packages related to test
2022-04-22 09:24:30 +02:00
Mole
7df4f84247 V10: Migrate member properties to columns on the member table (#12205)
* Add new columns to the member table

* Add missing IsApproved column

* Add joins with nested query

* Add query for selecting new column values from existing members

* Update the member data from the same query

* Make escapes work for sqlite

* Use GetFieldNameForUpdate instead of GetFieldName

* Left join on memberDto

* Remove the now unused property types and data

* Don't create member columns as properties anymore

* Store old properties in fields on member

Also switch the dates to nullable since they can be null

* Map columns when mapping from DTO to Member object

* Display columns in the member content app

* Fix null exception when creating new user

* Hide value if user doesn't have access to sensitive data

* Remove hardcoded member properties

* Obsolete old member alias constants

* Map and persist member properties

* Correctly update LastLogin when member logs in

* Map IsApproved and IsLockedOut when saving member in backoffice

* Update the query mappers for members

* Fix member service tracks dirty changes test

* Remove no longer existing property types from member type builder

* Fix null error in UpdateMemberProperties

* Fix builder tests

* Fix SetupMemberTestData in MemberControllerUnitTests

* Let LastLoginDate be null and handle check in controller

There's no reason to default a perfectly nullable property to default(DateTime)

* Add translation key for is approved and use that instead of constant

* Obsolete old label constants

* Fix whitespace post merge

* Fix up test comments

* Apply suggestions from code review

Co-authored-by: Elitsa Marinovska <21998037+elit0451@users.noreply.github.com>

* Fix member properties always being sensitive

Co-authored-by: Elitsa Marinovska <elm@umbraco.dk>
Co-authored-by: Elitsa Marinovska <21998037+elit0451@users.noreply.github.com>
2022-04-21 14:47:27 +02:00
Sebastiaan Janssen
6d0a0fff9c Merge branch 'v9/dev' into v10/dev 2022-04-21 14:37:28 +02:00
Mole
9b864ddbeb Fix member properties always being sensitive (#12282) 2022-04-21 13:51:24 +02:00
Nikolaj Geisle
060e0181a7 Merge remote-tracking branch 'origin/v10/dev' into v10/feature/nullable-reference-types-in-Umbraco.Web.Backoffice
# Conflicts:
#	src/Umbraco.Core/DependencyInjection/UmbracoBuilder.Collections.cs
#	src/Umbraco.Core/Extensions/PublishedContentExtensions.cs
#	src/Umbraco.Core/Telemetry/Models/TelemetryReportData.cs
#	src/Umbraco.Infrastructure/PropertyEditors/ValueConverters/MultiUrlPickerValueConverter.cs
#	src/Umbraco.PublishedCache.NuCache/ContentStore.cs
#	src/Umbraco.Web.BackOffice/Trees/MemberTypeTreeController.cs
#	src/Umbraco.Web.Common/ModelsBuilder/InMemoryModelFactory.cs
#	src/Umbraco.Web.Common/Security/MemberManager.cs
#	src/Umbraco.Web.Website/Routing/ControllerActionSearcher.cs
#	src/Umbraco.Web.Website/Routing/IControllerActionSearcher.cs
#	src/Umbraco.Web.Website/Routing/UmbracoRouteValueTransformer.cs
2022-04-21 10:26:51 +02:00
Bjarke Berg
2f1cb4b20a Fix build/tests + cleanup duplicate dependencies 2022-04-20 15:54:09 +02:00
Bjarke Berg
b19dab5f3a Fix merge issues 2022-04-20 15:42:27 +02:00
Bjarke Berg
23a49161d6 Merge remote-tracking branch 'origin/v9/dev' into v10/dev
# Conflicts:
#	build/build.ps1
#	build/templates/UmbracoPackage/.template.config/template.json
#	build/templates/UmbracoProject/.template.config/dotnetcli.host.json
#	build/templates/UmbracoProject/.template.config/ide.host.json
#	build/templates/UmbracoProject/.template.config/template.json
#	src/Umbraco.Core/Constants-System.cs
#	src/Umbraco.Infrastructure/DependencyInjection/UmbracoBuilder.Services.cs
#	src/Umbraco.Infrastructure/Security/BackOfficeUserStore.cs
#	src/Umbraco.Web.BackOffice/Controllers/AuthenticationController.cs
#	umbraco-netcore-only.sln
2022-04-20 11:09:28 +02:00
Nikolaj
ce6a3d6751 Fix TelemetryServiceTests 2022-04-19 15:19:25 +02:00
Nikolaj Geisle
c07ffb68fc v9: Implement telemetry levels (#12267)
* Add initial classes

* Add TelemetryProviders

* Add new NodeCountService.cs and NodeTelemetryProvider

* Add data contract attribute to UsageInformation

Otherwise it wont serialize correctly

* Implement more providers

* Fix builders and propertyEditorTelemetry

* Add MediaTelemetryProvider

* Add MediaTelemetryProvider

* Fix doubling of media telemetry

* Move contentCount from NodeCountTelemetryProvider and move to ContentTelemetryProvider

* Revert ContentTelemetryProvider changes

* Add detailed information to TelemetryService

* Add integration tests

* Add more tests and todos for tests

* Fix stylecop warnings

* Use yield return instead of instantiating local list

* Implement Macro test

* Inject interface instead of implementation in TelemetryService

* Fix TelemetryServiceTests.cs

* Implement media tests

* Implement propertyTypeTests

* Implement constants instead of hardcoded strings

* Add SystemInformationTelemetryProvider

* Use SystemInformationTableDataProvider in UserDataService

* Implement more properties

* Add UsageInformation

* Replace UserDataService with SystemInformationTelemetryProvider

* Undo changes to UserDataService and obsolete it

* Remove ISystemInformationTableDataProvider

* Register SystemInformationTelemetryProvider as telemetry provider

* Use constants for telemetry names

* Make UserDataServiceTests test SystemInformationTelemetryProvider instead

* Update UserDataServiceTests to cover new data

* Add unit tests

* Add integration test testing expected data is returned

* Implement Analytics dashboard

* Improve assertion message

* Add text and styling to analyticspage

* Rename consent to analytic

* implement save button for consent level

* Implement save button

* Fix system information test

* Add TelemetryResource

* Move telemetry providers to infrastructure

* Add database provider to system information

* Set startvalue for slider

* Fix unit tests

* Implement MetricsConsentService using KeyValueService

* Return void hen setting the telemetry level

* fix startposition when not reloading

* Add a couple tests

* Update src/Umbraco.Core/Services/MetricsConsentService.cs

* Rename ConsentLevel.cs

* Use direct Enum instead of parsing

* rename consent resource

* add lazy database

* refactor slider

* Implement ng-if and propers pips

* Make classes internal

* Fix slider not loading when navigating to tab

* Add telemetry level check to TelemetryService.cs

* Add Consent for analytics text

* Fix build errors for unit tests

* Fix TelemetryServiceTests

* revert package-lock.json

* Fix integration test

* Update slider

* Update TelemetryService.cs

* Apply suggestions from code review

Co-authored-by: Mole <nikolajlauridsen@protonmail.ch>

Co-authored-by: Nikolaj Geisle <niko737@edu.ucl.dk>
Co-authored-by: nikolajlauridsen <nikolajlauridsen@protonmail.ch>
2022-04-19 15:06:10 +02:00
Bjarke Berg
852305b7d1 Simplified setup of 2FA for users (#12142)
* Added functionality to enable 2FA for users..

* Do not use the obsolete ctor in tests

* cleanup

* Cleanup

* Convert User view from overlay to infinite editor

* Add support for having additional editors on top of the user (2fa) which overlay does not support
* Add controllerAs syntax in the template
* Remove unused dependencies

* Adjustments to 2fa login view

* organize elements
* add translations
* add a11y helpers
* add autocompletion = one-time-code
* change to controllerAs syntax

* add callback to cancel 2fa and fix error where submit button was not reset when all other validations were

* add a cancel/go back button to the 2fa view

* replace header with something less obstrusive

* move logout button to the footer in the new editor view

* change 'edit profile' to an umb-box and move ng-if for password fields out to reduce amount of checks

* Add umb-box to external login provider section

* add umb-box to user history section

* bug: fix bug where notificationsService would not allow new notifications if removeAll had been called

* add styling and a11y to configureTwoFactor view

- also ensure that the view reloads when changes happen in the custom user view to enable 2fa
- ensure that view updates when disabling 2fa
- add extra button to show options (disable) for each 2fa provider

* add notification when 2fa is disabled

* add data-element to support the intro tour

also changed a minor selector in the cypress test

* correct usage of umb-box with umb-box-content

* do not use the .form class twice to prevent double box-shadow

* make tranlastion for 2fa placeholder shorter

* ensure that field with 2fa provider is always visible when more than 1 provider

* move error state of 2fa field to token field

* update translation of multiple 2fa providers

* move CTA buttons to right side to follow general UI practices

* rename options to disable

* add disabled state

* add helper folders to gitignore so you can work with plugins and custom code without committing it accidentally

* move the disable functionality to its own infinite editor view

* use properties from umb-control-group correctly

* add 'track by' to repeater

* make use of umb-control-group

* remove unused functions

* clean up translations

* add Danish translations

* copy translations to english

* Only return enabled 2fa providers as expected

Co-authored-by: Jacob Overgaard <752371+iOvergaard@users.noreply.github.com>
2022-04-19 08:33:03 +02:00
Sebastiaan Janssen
48dbe959f6 Merge pull request #12249 from umbraco/v9/feature/merge-v9-contrib 2022-04-14 11:44:24 +02:00
Sebastiaan Janssen
75613cf061 Revert "Block List Settings throws exception if Models builder mode is set to "Nothing" (#11725)"
This reverts commit 62b289e179.
2022-04-12 13:45:15 +02:00
Sebastiaan Janssen
4e71d6f2f6 Merge branch 'v9/dev' into v9/contrib
# Conflicts:
#	src/Umbraco.Infrastructure/HostedServices/RecurringHostedServiceBase.cs
2022-04-12 13:41:34 +02:00
Ronald Barendse
c0b0a47612 Merge branch 'v9/dev' into v9/feature/projecttemplate-cleanup 2022-04-12 11:16:16 +02:00
Nikolaj Geisle
8865c5d486 Merge remote-tracking branch 'origin/v10/dev' into v10/feature/nullable-reference-types-in-Umbraco.Web.Backoffice 2022-04-05 11:25:52 +02:00
Paul Johnson
d036c6a118 Fix tests when running filtered set (SQL Server)
Without InstallDefaultDataSettings DatabaseSchemaCreator makes use of
static service locator which isn't wired up in time.
2022-04-05 10:18:04 +01:00
Nikolaj Geisle
76c525b611 Merge remote-tracking branch 'origin/v10/dev' into v10/feature/nullable-reference-types-in-Umbraco.Web.Backoffice 2022-04-05 11:11:44 +02:00
Paul Johnson
c34479c2bb Fix tests when running filtered set.
Without InstallDefaultDataSettings DatabaseSchemaCreator makes use of
static service locator which isn't wired up in time.
2022-04-05 10:04:39 +01:00
Nikolaj Geisle
a6128bfe8e Merge branch 'v10/dev' into v10/feature/nullable-reference-types-in-Umbraco.Web.Backoffice
# Conflicts:
#	src/Umbraco.Core/Cache/MacroCacheRefresher.cs
#	src/Umbraco.Core/Services/MacroService.cs
#	src/Umbraco.Core/StaticApplicationLogging.cs
#	src/Umbraco.Infrastructure/Migrations/Install/DatabaseDataCreator.cs
#	src/Umbraco.Infrastructure/Persistence/Repositories/Implement/MacroRepository.cs
#	src/Umbraco.Infrastructure/Persistence/Repositories/Implement/TrackedReferencesRepository.cs
#	src/Umbraco.Infrastructure/PropertyEditors/GridPropertyEditor.cs
#	src/Umbraco.Infrastructure/Security/UmbracoPasswordHasher.cs
#	src/Umbraco.Web.BackOffice/Controllers/ImagesController.cs
2022-04-05 08:51:08 +02:00
Nikolaj Geisle
4710951185 fix build errors 2022-04-04 12:45:06 +02:00
dependabot[bot]
ddddaeb113 Bump minimist from 1.2.5 to 1.2.6 in /tests/Umbraco.Tests.AcceptanceTest
Bumps [minimist](https://github.com/substack/minimist) from 1.2.5 to 1.2.6.
- [Release notes](https://github.com/substack/minimist/releases)
- [Commits](https://github.com/substack/minimist/compare/1.2.5...1.2.6)

---
updated-dependencies:
- dependency-name: minimist
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-04-04 08:46:29 +10:00
Nikolaj Geisle
70a00901b4 Finish up Web.Backoffice 2022-04-01 14:35:18 +02:00
Bjarke Berg
fe1c2f5d4f Merge remote-tracking branch 'origin/v9/dev' into v10/dev
# Conflicts:
#	build/templates/UmbracoPackage/.template.config/template.json
#	build/templates/UmbracoProject/.template.config/template.json
#	src/Directory.Build.props
#	src/Umbraco.Core/DependencyInjection/UmbracoBuilder.Configuration.cs
#	src/Umbraco.Infrastructure/DependencyInjection/UmbracoBuilder.Repositories.cs
#	src/Umbraco.Infrastructure/DependencyInjection/UmbracoBuilder.Services.cs
#	src/Umbraco.Infrastructure/HostedServices/HealthCheckNotifier.cs
#	src/Umbraco.Infrastructure/HostedServices/RecurringHostedServiceBase.cs
#	src/Umbraco.Infrastructure/HostedServices/ServerRegistration/TouchServerTask.cs
#	src/Umbraco.Infrastructure/Migrations/Install/DatabaseSchemaCreator.cs
#	src/Umbraco.Infrastructure/Persistence/Repositories/Implement/TrackedReferencesRepository.cs
#	src/Umbraco.Web.Common/Middleware/UmbracoRequestMiddleware.cs
#	tests/Umbraco.Tests.Integration.SqlCe/Umbraco.Infrastructure/Persistence/DatabaseBuilderTests.cs
#	tests/Umbraco.Tests.Integration/Testing/BaseTestDatabase.cs
#	tests/Umbraco.Tests.Integration/Umbraco.Infrastructure/Migrations/AdvancedMigrationTests.cs
#	tests/Umbraco.Tests.Integration/Umbraco.Infrastructure/Persistence/SchemaValidationTest.cs
#	tests/Umbraco.Tests.Integration/Umbraco.Infrastructure/Persistence/SqlServerTableByTableTest.cs
#	tests/Umbraco.Tests.UnitTests/Umbraco.Core/Components/ComponentTests.cs
2022-03-31 12:51:55 +02:00
Andy Butland
02e76c8227 Configuration to control the creation of default data (#12122)
* Added configuration and checks for creation of default Umbraco data.

* Fixed configuration binding issues.

* Updated comments.

* Added DefaultDataCreationSettings to the JSON schema.

* Removed option to not install default relation types as Umbraco relies on (and will recreate) them if they aren't there.

* Renamed configuration class used for install of default data and converted to named optios.

* Fix to failing unit tests.

* Fixes for integration tests.

* Apply suggestions from code review

Co-authored-by: Elitsa Marinovska <21998037+elit0451@users.noreply.github.com>

* Further fix from code review.

* Updated naming as per PR review suggestions.

* Update src/Umbraco.Infrastructure/Migrations/Install/DatabaseDataCreator.cs

Co-authored-by: Elitsa Marinovska <21998037+elit0451@users.noreply.github.com>

Co-authored-by: Elitsa Marinovska <21998037+elit0451@users.noreply.github.com>
2022-03-25 08:58:07 +01:00
Ronald Barendse
c511f19cc8 Merge branch 'v9/feature/move-templates-to-root' into v9/feature/projecttemplate-cleanup 2022-03-24 13:57:38 +01:00
Nikolaj Geisle
a02ef3c101 Try to make test non-flaky 2022-03-18 14:52:57 +01:00
Nikolaj Geisle
181b984a28 Fix cypress test 2022-03-17 10:25:59 +01:00
Elitsa Marinovska
c08921d3e6 Merge remote-tracking branch 'origin/v9/dev' into v9/bugfix/track-media-items-picked-as-macro-params 2022-03-16 15:11:58 +01:00
Nikolaj Geisle
214bcceffc Merge v10/dev into infrastructure 2022-03-16 14:39:28 +01:00
Elitsa Marinovska
5bebf98339 Test + fix 2022-03-16 14:19:25 +01:00
Nikolaj Geisle
4ebd20bb66 Merge remote-tracking branch 'origin/v10/dev' into v10/feature/nullable-reference-types-in-Umbraco-Core
# Conflicts:
#	build/build.ps1
#	src/Umbraco.Core/Configuration/ConfigConnectionString.cs
#	src/Umbraco.Core/Configuration/Models/ConnectionStrings.cs
#	src/Umbraco.Core/Install/InstallSteps/TelemetryIdentifierStep.cs
#	src/Umbraco.Core/Models/ContentType.cs
#	src/Umbraco.Infrastructure/Migrations/Install/DatabaseBuilder.cs
#	tests/Umbraco.Tests.AcceptanceTest/package.json
2022-03-16 13:00:38 +01:00
Paul Johnson
3961c4c233 v10 SQLite support + distributed locking abstractions (#11922)
* Created Persistence.SQLite project skeleton.

* SQLite database initialization

* Various changes and hacks to make things work.

* WIP integration tests

* Fix thread safety tests

* Fix tests that relied on tie breaker sorting.

Spent a fair amount of time looking for a less lazy fix but gave up.

* Convert right join to left join ContentTypeRepository.PerformGetByQuery
SQLite doesn't support right join

* Fix test Can_Generate_Delete_SubQuery_Statement

Worth noting that NPoco.DatabaseTypes.SQLiteDatabaseType doesn't override
EscapeSqlIdentifier so NPoco will escape with [].

SQLite docs say > "A keyword enclosed in square brackets is an identifier.
This is not standard SQL.
This quoting mechanism is used by MS Access and SQL Server and is
included in SQLite for compatibility."

Also could have updated SqliteSyntaxProvider to match npoco but
decided against it.

* Fixes for paginated custom order by

* Fix tests broken by lack of unique indexes.

* Fix SqlServerTableByTableTest tests.

These tests didn't actually do anything as the tables already exist so schema creator just returned.

Did however point out that the default implementation for DoesTableExist just returns false so added a default naive implementation.

* Fix ValidateLoginSession - SelectTop must come later

* dry up database cleanup

* Fix up db migration tests.
We can't drop pk in sqlite without recreating table.
Test looks to be testing that add column works as intended which we can test.

* Prevent schema creation errors.

* SQLite ignore lock tests, WAL back on.

* Fix package schema tests

* Fix NPocoFetchTests - case sensitivity not under test

* Fix AdvancedMigrationTests (where possible)

Migrations probably need a good look later.
Maybe nuke old migrations and only support moving to v10 from v9.

If we do that can do some cleanup.

* Cleanup test database configuration

* Run integration tests against SQLite on build agent.

* Drop MS.Data.SQLite
System.Data.SQLite was quicker to roll out due to more CLR type mapping

* YAML

* Skip Umbraco.Tests.Integration.SqlCe

* Drop SqlServerTableByTable tests.

Until this week they did nothing anyway as they with NewSchemaPerTest
so the tests all passed as CreateTable was no op (already exists).

Also all of the tables are created in an empty database by SchemaValidationTest.cs
DatabaseSchemaCreation_Produces_DatabaseSchemaResult_With_Zero_Errors

* Might aswell run against macOS also.

* Copy azure pipelines task header layout

* Delete SQLCe projects

* Remove SQL CE specific code.

* Remove SQL CE NuSpec, template params, build script setup

* Delete umbraco-netcore-only.sln

* Add SkipTests solution configuration and use for codeql

* Remove reference to deleted nuspec file.

* Refactor ConnectionStrings WRT DataDirectory placeholder & ProviderName.

At this point you can try out SQLite support by setting the following
in appsettings.json and then completing the install process.

"ConnectionStrings": {
  "umbracoDbDSN": "Data Source=|DataDirectory|/umbraco.sqlite",
  "umbracoDbDSN_ProviderName": "System.Data.SQLite"
},

Not currently possible via installer UI without provider name pre-set in
configuration.

* Switch to Microsoft.Data.Sqlite

Some gross hacks but will be good to find out if this works
with apple silicon.

* Enable selection of SQLite via installer UI (also quick install)

* Remove SqlServerDbProviderFactoryCreator to cleanup a TODO

* Move SQL Server support to its own class library

* Add persistence dependencies to Umbraco.CMS metapackage

* Bugfix packages delete query

Created invalid query for SQLite.

* Try out cypress tests Linux + SQLite

* Prevent cypress test artifact upload failure on attempt 2+

* LocalDb bugfixes

* Drop redundant enum

* Move SqlClient constant

* Misc whitespace

* Remove IsSqlCe extension (TODO: drop non 9->10 migrations later).

* Umbraco.Persistence.* -> Umbraco.Cms.Persistence.*

* Display quick install defaults and per provider default database name.

* Misc remove old comment

* little re-arrange

* Remove almost all usages of IsSqlite extension.

* visual adjustments

* Custom Database Configuration is last step and should then say Install.

* use text instead of disabled inputs

* move legend, rename to Install

* Update SqlMainDomLock to work without distributed locks.

* Added IDistributedLockingMechanism interface and in memory impl.

* Drop locking from ISqlSyntaxProvider & wire up scope to abstraction.

* Added SqlServerDistributedLockingMechanism

* Move distributed locking interfaces and exceptions to Core + xmldocs.

* Fix tests, Misc cleanup, Add SQL distributed locking integration tests

* Provide mechanism to specify DistributedLockingMechanism in config
(even if added by composer)

* Nomplementation -> NoImplementation

* Fix misleading comment

* Integration tests use SqlServerDistributedLockingMechanism when possible

* Handle up-gradable locks SqlServerDistributedLockingMechanism.
TODO: InMemoryDistributedLockingMechanism.

Note: Nuked SqlServerDistributedLockingMechanismTests, will still sleep
at night.
Is covered by Umbraco.Cms.Tests.Integration.Umbraco.Infrastructure.Persistence.LockTests

* Make tests pass for InMemoryDistributedLockingMechanism, pretty hacky.

* Tweak constraints on WithCollectionBuilder so i can drop bad constructor

* Added SqliteDistributedLockingMechanism

* Dropped InMemoryDistributedMechanism + magic

InMemoryDistributedMechanism was pretty rubbish and now we have
a decent implementation for SQLite as we no longer block readers
see 8d1f42b.

Also drop the CollectionBuilder setup, instead do the same as we do
for syntax providers etc, it's more automagical so we never require an
explicit selection although we are allowing for it.

However keeping the optional IUmbracoBuilder constructor param for
CollectionBuilders as it's extremely useful.

* Fix quick install "" database name.

* Hide Database Configuration section when a connection string is pre-set.

Doesn't seem worth it to extract db name from connection string.

* Ensure wal test 2+

* Fix logging inconsistencies.

* Ensure in transaction when obtaining locks + no-op the SQLite read lock.

There's no point in running the query just to make a single test pass.

* Fix installer database display names

* Allow SQLite shared cache without losing deferred transactions

* Opt into shared cache for new SQLite databases + fix filename

* Fix misc inconsistency in .gitignore

* Prefer our interceptor interface

* Restore DEBUG_DATABASES code OnConnectionOpened in case it's used.

* Back to private cache.

* Added retry strategy for SQLite + refactor out SQL server specific stuff

* Fix SQL server tests.

* Misc - Orphaned comment, incorrect casing.

* InMemory SQLite test database & turn shared cache back on everywhere.

Co-authored-by: Niels Lyngsø <niels.lyngso@gmail.com>
2022-03-11 17:14:20 +01:00