Commit Graph

1135 Commits

Author SHA1 Message Date
Shannon Deminick
eb70bc313a Implements Examine/Lucene index syncing for use on Azure (#10386)
* Updates to latest examine, ensures indexes are unlocked on startup, adds more info to the diagnostics and allows for the new sync directory factory to work.

* Gets index syncing working correct and fixes reindexing to not overprocess indexes

* fix duplicate package ref

* rebuilds empty indexes and fixes config with enum

* missing file
2021-06-07 14:58:49 +02:00
Shannon
3c3c9a6a47 Merge remote-tracking branch 'origin/v8/contrib' into v9/dev
# Conflicts:
#	src/Umbraco.Core/Sync/DatabaseServerMessenger.cs
#	src/Umbraco.Core/Sync/DatabaseServerMessengerOptions.cs
#	src/Umbraco.Core/Sync/ISyncBootStateAccessor.cs
#	src/Umbraco.Core/Sync/NonRuntimeLevelBootStateAccessor.cs
#	src/Umbraco.Core/Sync/SyncBootState.cs
#	src/Umbraco.Tests/PublishedContent/NuCacheChildrenTests.cs
#	src/Umbraco.Tests/PublishedContent/NuCacheTests.cs
#	src/Umbraco.Tests/Scoping/ScopedNuCacheTests.cs
#	src/Umbraco.Tests/Services/ContentTypeServiceVariantsTests.cs
#	src/Umbraco.Web.UI.Client/package.json
#	src/Umbraco.Web.UI/Umbraco/config/lang/da.xml
#	src/Umbraco.Web.UI/Umbraco/config/lang/de.xml
#	src/Umbraco.Web.UI/Umbraco/config/lang/en.xml
#	src/Umbraco.Web.UI/Umbraco/config/lang/en_us.xml
#	src/Umbraco.Web.UI/Umbraco/config/lang/es.xml
#	src/Umbraco.Web.UI/Umbraco/config/lang/fr.xml
#	src/Umbraco.Web.UI/Umbraco/config/lang/pl.xml
#	src/Umbraco.Web.UI/Umbraco/config/lang/ru.xml
#	src/Umbraco.Web/BatchedDatabaseServerMessenger.cs
#	src/Umbraco.Web/Compose/DatabaseServerRegistrarAndMessengerComponent.cs
#	src/Umbraco.Web/HealthCheck/Checks/Config/AbstractConfigCheck.cs
#	src/Umbraco.Web/HealthCheck/Checks/Config/NotificationEmailCheck.cs
#	src/Umbraco.Web/HealthCheck/HealthCheckAction.cs
#	src/Umbraco.Web/PublishedCache/NuCache/NuCacheComposer.cs
#	src/Umbraco.Web/PublishedCache/NuCache/PublishedSnapshotService.cs
#	src/Umbraco.Web/Search/ExamineFinalComponent.cs
2021-06-04 10:59:52 -07:00
Matthew-Wise
c341a39306 Add a collection to allow adding CSS and JS to the back office via C# (#10310)
* Added Custom back office collection builder, to allow css / js to be added to the back office using c#

* Added call to the collection

* Added obselete attribute to PropertyEditorAssetAttribute
2021-06-02 12:28:14 +02:00
Mole
40c8bf62f7 Enable email sender to publish an event if smtp is not configured (#10352) 2021-06-01 06:41:56 +02:00
Bjarke Berg
8a0a6e62bf Merge remote-tracking branch 'origin/v8/dev' into v9/dev
# Conflicts:
#	src/SolutionInfo.cs
#	src/Umbraco.Core/Cache/DataTypeCacheRefresher.cs
#	src/Umbraco.Core/Extensions/PublishedContentExtensions.cs
#	src/Umbraco.Core/Extensions/PublishedPropertyExtension.cs
#	src/Umbraco.Core/Migrations/Install/DatabaseDataCreator.cs
#	src/Umbraco.Tests/TestHelpers/Entities/MockedContentTypes.cs
#	src/Umbraco.Web.UI.NetCore/umbraco/config/lang/da.xml
#	src/Umbraco.Web/PropertyEditors/MediaPickerPropertyEditor.cs
2021-05-31 15:10:49 +02:00
Bjarke Berg
7a63e26785 Add missing IMigrationContext to RebuildPublishedSnapshot ctor 2021-05-26 18:31:03 +02:00
Callum Whyte
b07f6519e7 Rename models builder modes (#10272)
* Renaming AppData Models Builder mode

* Renaming PureLive Models Builder mode to Runtime

* ModelsBuilderAssembly attribute flags if models are "Live" or not

* Updated ModelsMode names

* Only add tag if it is not a pull request. Apparently this is not allowed when it is from a fork.

* Revert "Only add tag if it is not a pull request. Apparently this is not allowed when it is from a fork."

This reverts commit 92b33f3c

Co-authored-by: Bjarke Berg <mail@bergmania.dk>
2021-05-26 08:36:21 +02:00
Bjarke Berg
3b8f9d5e33 https://github.com/umbraco/Umbraco-CMS/issues/10295 - Use FileUploadConfigurationEditor in FileUploadPropertyEditor 2021-05-25 08:53:13 +02:00
Bjarke Berg
3aaf53ce0e Merge pull request #10299 from umbraco/v9/feature/allow-custom-notification-publisher-in-scope
V9: Allow custom notification publisher in scope
2021-05-21 11:07:32 +02:00
Bjarke Berg
9aee6919d7 Merge pull request #10286 from umbraco/v9/feature/do_not_restart_doing_simple_installs_without_package_installation
V9: Do not restart doing simple installs without package installation
2021-05-20 21:36:55 +02:00
Bjarke Berg
9b2cf8422d Removed unused ctor (just for tests) + Fixed a couple of handlers that is not intended to execute before runtime level = run. 2021-05-20 13:16:50 +02:00
Bjarke Berg
0a3b39cc38 Merge remote-tracking branch 'origin/v9/dev' into v9/feature/do_not_restart_doing_simple_installs_without_package_installation 2021-05-20 12:56:41 +02:00
Mole
6596bcf557 Inject notification publisher from CreateScope 2021-05-20 12:36:57 +02:00
Bjarke Berg
025a7aca29 Updated nuget packages 2021-05-19 21:44:33 +02:00
Bjarke Berg
80ee7b398c Acquire the maindom before we do unattended installs 2021-05-19 20:40:06 +02:00
Bjarke Berg
8131d1a9c9 Added xml docs 2021-05-18 19:33:54 +02:00
Bjarke Berg
da895ae65b Added RestartAsync to IRuntime instead of cancellation token 2021-05-18 19:30:07 +02:00
Bjarke Berg
4c78702da0 Do not restart after install, instead stop and restart the runtime. Also signin the admin user after install. 2021-05-18 19:19:58 +02:00
Bjarke Berg
65b20572b7 V9: Resolve virtual view paths from DataEditorAttribute in DataValueEditor (#10279)
* https://github.com/umbraco/Umbraco-CMS/issues/10265
Resolve virtual view paths from DataEditorAttribute in DataValueEditor + Introduced an IDataValueEditorFactory, so we don't need to inject nested dependencies every time we introduce a new dependency in DataValueEditor..

* Cleanup + xml doc
2021-05-18 12:40:24 +02:00
Shannon Deminick
eba6373a12 Examine 2.0 integration (#10241)
* Init commit for examine 2.0 work, most old umb examine tests working, probably a lot that doesn't

* Gets Umbraco Examine tests passing and makes some sense out of them, fixes some underlying issues.

* Large refactor, remove TaskHelper, rename Notifications to be consistent, Gets all examine/lucene indexes building and startup ordered in the correct way, removes old files, creates new IUmbracoIndexingHandler for abstracting out all index operations for umbraco data, abstracts out IIndexRebuilder, Fixes Stack overflow with LiveModelsProvider and loading assemblies, ports some changes from v8 for startup handling with cold boots, refactors out LastSyncedFileManager

* fix up issues with rebuilding and management dashboard.

* removes old files, removes NetworkHelper, fixes LastSyncedFileManager implementation to ensure the machine name is used, fix up logging with cold boot state.

* Makes MainDom safer to use and makes PublishedSnapshotService lazily register with MainDom

* lazily acquire application id (fix unit tests)

* Fixes resource casing and missing test file

* Ensures caches when requiring internal services for PublishedSnapshotService, UseNuCache is a separate call, shouldn't be buried in AddWebComponents, was also causing issues in integration tests since nucache was being used for the Id2Key service.

* For UmbracoTestServerTestBase enable nucache services

* Fixing tests

* Fix another test

* Fixes tests, use TestHostingEnvironment, make Tests.Common use net5, remove old Lucene.Net.Contrib ref.

* Fixes up some review notes

* Fixes issue with doubly registering PublishedSnapshotService meanig there could be 2x instances of it

* Checks for parseexception when executing the query

* Use application root instead of duplicating functionality.

* Added Examine project to netcore only solution file

* Fixed casing issue with LazyLoad, that is not lowercase.

* uses cancellationToken instead of bool flag, fixes always reading lastId from the LastSyncedFileManager, fixes RecurringHostedServiceBase so that there isn't an overlapping thread for the same task type

* Fix tests

* remove legacy test project from solution file

* Fix test

Co-authored-by: Bjarke Berg <mail@bergmania.dk>
2021-05-18 10:31:38 +02:00
Andy Butland
01643e494a Removed unecessary second calls to retrieve the file system in TemplateRepository. 2021-05-12 14:53:43 +02:00
Andy Butland
fd9faf5a99 Re-implemented various file service methods from V8 required in Deploy. 2021-05-12 11:58:36 +02:00
Bjarke Berg
de0929762c Fix issue with build number as part of the version 2021-05-12 09:49:41 +02:00
Mole
2bf86acf38 V9: Place notifications in the same namespace (#10231)
* Gather all notifications in Umbraco.Cms.Core.Notifications

* Rename notifications to match convention

* Move and rename missed notifications

* Move the three remaining public notification into Umbraco.Cms.Core.Notifications
2021-05-11 14:33:49 +02:00
Mole
11ca2c5d04 Merge pull request #10262 from umbraco/v9/feature/validate-credentials
Creates a method allowing validation of a user's credentials without actually logging them in.
2021-05-11 13:42:34 +02:00
Bjarke Berg
fe36c80ba0 Merge remote-tracking branch 'origin/v8/dev' into v9/dev
# Conflicts:
#	src/Umbraco.Core/Cache/UserGroupCacheRefresher.cs
#	src/Umbraco.Core/Migrations/Install/DatabaseDataCreator.cs
#	src/Umbraco.Core/Persistence/Repositories/Implement/RepositoryCacheKeys.cs
#	src/Umbraco.Core/PropertyEditors/MediaPickerWithCropsValueConverter.cs
#	src/Umbraco.Core/PropertyEditors/Validators/FileExtensionConfigItem.cs
#	src/Umbraco.Core/PropertyEditors/Validators/FileUploadConfiguration.cs
#	src/Umbraco.Core/PropertyEditors/Validators/FileUploadConfigurationEditor.cs
#	src/Umbraco.Core/PropertyEditors/Validators/IFileExtensionConfig.cs
#	src/Umbraco.Core/PropertyEditors/Validators/IFileExtensionConfigItem.cs
#	src/Umbraco.Core/PropertyEditors/Validators/MediaPicker3Configuration.cs
#	src/Umbraco.Core/PropertyEditors/Validators/MediaPicker3ConfigurationEditor.cs
#	src/Umbraco.Core/PropertyEditors/Validators/MediaPicker3PropertyEditor.cs
#	src/Umbraco.Infrastructure/Cache/DefaultRepositoryCachePolicy.cs
#	src/Umbraco.Infrastructure/PropertyEditors/ValueConverters/JsonValueConverter.cs
#	src/Umbraco.Tests.Integration/Umbraco.Infrastructure/Persistence/Repositories/MediaTypeRepositoryTest.cs
#	src/Umbraco.Tests/Composing/TypeLoaderTests.cs
#	src/Umbraco.Tests/PublishedContent/PublishedMediaTests.cs
#	src/Umbraco.Tests/Services/MediaServiceTests.cs
#	src/Umbraco.Web.BackOffice/Controllers/MediaController.cs
#	src/Umbraco.Web.Common/Extensions/FriendlyImageCropperTemplateExtensions.cs
#	src/Umbraco.Web.UI.Client/src/views/propertyeditors/fileupload/fileupload.controller.js
#	src/Umbraco.Web.UI.NetCore/umbraco/config/lang/da.xml
#	src/Umbraco.Web.UI.NetCore/umbraco/config/lang/en.xml
#	src/Umbraco.Web.UI.NetCore/umbraco/config/lang/en_us.xml
#	src/Umbraco.Web/Cache/MemberCacheRefresher.cs
#	src/Umbraco.Web/PropertyEditors/FileUploadPropertyEditor.cs
#	src/Umbraco.Web/UrlHelperRenderExtensions.cs
2021-05-11 12:33:31 +02:00
Andy Butland
3e19824be2 Creates a method available in V8 allowing validation of a user's credentials without actually logging them in. 2021-05-11 11:29:40 +02:00
Bjarke Berg
145a0fabe7 Fixed issues where RuntimeState.Level was used in ctors. This is not a good idea, as these ctors are executed before the level are determined. 2021-05-07 07:40:08 +02:00
Warren Buckley
cb01e11586 Deterministic Builds for Sourcelink (#10237)
* Remove the properties from each .CSProj

* Move this into Directory.Build.props so all CSProj can get all these extra properties

* Add conditional to only add extra property to get the Deterministic green checkmark when running on Azure Pipleine
2021-05-06 19:56:43 +02:00
Bjarke Berg
205c111dbc https://github.com/umbraco/Umbraco-CMS/issues/10232
Fixes issue with modelsbuilder when doctypes are inheriting from other doc types.
2021-05-06 10:58:45 +02:00
Warren Buckley
0bdeda378a Merge pull request #10205 from umbraco/v9/feature/replace-system-drawing-with-imagesharp
Replaces System.Drawing with ImageSharp, and thereby fix TODO.
2021-05-05 14:57:01 +01:00
Warren Buckley
f43f37dc35 Umbraco V9 Adds SourceLink package from Microsoft for people to debug into the relevent build commit with GitHub source code (#10224)
We had this previously in Umbraco V8
2021-05-04 18:58:50 +02:00
Martin Amsinck Andersen
2bc1c46e57 Casing in appsettings:umbraco:cms:contentMacroErrors and using MacroErrorBehaviour Enum (#10208)
* appsettings "WebRouting:DisableRedirectUrlTracking" should be a bool not a string

* Casing in appsettings "Umbraco:CMS:Content:MacroErrors" + Using MacroErrorBehaviour enum
2021-05-04 13:40:53 +02:00
Elitsa Marinovska
4bdb011b16 V9: Implementing Error Describers for translated error messages (#10216)
* Adding missing keys needed for the translation of the error describers' msgs

* Override all methods from IdentityErrorDescriber in order to provide our own translated error messages using our own translation engine (ILocalizedTextService)

* Fix tests

* Refactoring based on introducing an UmbracoErrorDescriberBase

* Remove unnecessary usings
2021-05-03 14:05:17 +02:00
Andy Butland
12c55dd605 Made the LocalDb class public for access from Deploy. 2021-04-30 16:45:26 +02:00
Andy Butland
1f8810ebb2 Made some further DTOs public so can be used in Deploy. 2021-04-30 16:20:53 +02:00
Bjarke Berg
a854959b85 Replaces System.Drawing with ImageSharp, and thereby fix TODO. 2021-04-29 20:32:26 +02:00
Andy Butland
b12c4b7d66 Aligned access modifier of related Dto. 2021-04-29 16:52:18 +02:00
Andy Butland
4ddc6ad868 Made a couple of internal classes public that Deploy has a dependency on. 2021-04-29 15:27:02 +02:00
Kenn Jacobsen
3781c6b00f Remove a bunch of template related, unreferenced methods from file service and stylesheet repository (#9875)
Co-authored-by: Bjarke Berg <mail@bergmania.dk>
2021-04-27 12:30:38 +02:00
Kenn Jacobsen
8674244e61 Remove a bunch of partial views related, unreferenced methods from file service and partial view repositories (#9879)
Co-authored-by: Bjarke Berg <mail@bergmania.dk>
2021-04-27 11:42:06 +02:00
Kenn Jacobsen
4f0d4d1fcd Remove a bunch of script related, unreferenced methods from file service and script repository (#9878)
Co-authored-by: Bjarke Berg <mail@bergmania.dk>
2021-04-27 10:36:28 +02:00
Kenn Jacobsen
720e529474 Remove a bunch of template related, unreferenced methods from file service and template repository (#9876) 2021-04-27 09:59:51 +02:00
Mole
c9ebaadf23 Netcore: File systems rework (#10181)
* Allow IMediaFileSystem to be replace in the DI, or registered with inner filesystem

* Remove GetFileSystem from Filesystems

It was only used by tests.

* Make MediaFileSystem inherit from PhysicalFileSystem directly

* Remove FileSystemWrapper

* Remove inner filesystem from MediaFileSystem

* Add MediaFileManager and bare minimum to make it testable

* Remove MediaFileSystem

* Fix unit tests using MediaFileManager

* Remove IFileSystem and rely only on FileSystem

* Hide dangerous methods in FileSystems and do some cleaning

* Apply stylecop warnings to MediaFileManager

* Add FilesystemsCreator to Tests.Common

This allows you to create an instance if FileSystems with your own specified IFileSystem for testing purposes outside our own test suite.

* Allow the stylesheet filesystem to be replaced.

* Fix tests

* Don't save stylesheetWrapper in a temporary var

* refactor(FileSystems): change how stylesheet filesystem is registered

* fix(FileSystems): unable to overwrite media filesystem

SetMediaFileSystem added the MediaManager as a Singleton instead of
replacing the existing instance.

* fix(FileSystems): calling AddFileSystems replaces MediaManager

When calling AddFileSystems after SetMediaFileSystem the MediaManager
gets replaced by the default PhysicalFileSystem, so instead of calling
SetMediaFileSystem in AddFileSystems we now call TrySetMediaFileSystem
instead. This method will not replace any existing instance of the
MediaManager if there's already a MediaManager registered.

* Use SetMediaFileSystem instead of TrySet, and rename AddFilesystems to ConfigureFileSystems

Also don't call AddFileSystems again in ConfigureFilesystems

* Don't wrap CSS filesystem twice

* Add CreateShadowWrapperInternal to avoid casting

* Throw UnauthorizedAccessException isntead of InvalidOperationException

* Remove ResetShadowId

Co-authored-by: Rasmus John Pedersen <mail@rjp.dk>
2021-04-27 09:52:17 +02:00
Mole
51a720b16c Add references to Umbraco.Code as PrivateAsset (#10148)
Co-authored-by: Bjarke Berg <mail@bergmania.dk>
2021-04-27 09:07:10 +02:00
Andy Butland
7dd92efe25 Added additional addresses and attachments to core's email sending service. (#10178) 2021-04-23 08:59:10 +02:00
Shannon Deminick
9382500b1f Ensure we don't issue a write lock on member login (#10175)
If only the last login property has changed we can issue a SQL call without write locks to make that update.
2021-04-22 18:37:56 +02:00
Shannon Deminick
57009be435 Stop allocating arrays for every scope lock (#10172)
This is left over from v8's APIs where we would pass in params int[] but in most cases we are just passing in a single value yet we are then allocating arrays everytime. There's no need to allocate these, it's just extra overhead.
2021-04-22 18:36:45 +02:00
Shannon Deminick
39aeec0f1f Implement password config storage for members (#10170)
* Getting new netcore PublicAccessChecker in place

* Adds full test coverage for PublicAccessChecker

* remove PublicAccessComposer

* adjust namespaces, ensure RoleManager works, separate public access controller, reduce content controller

* Implements the required methods on IMemberManager, removes old migrated code

* Updates routing to be able to re-route, Fixes middleware ordering ensuring endpoints are last, refactors pipeline options, adds public access middleware, ensures public access follows all hops

* adds note

* adds note

* Cleans up ext methods, ensures that members identity is added on both front-end and back ends. updates how UmbracoApplicationBuilder works in that it explicitly starts endpoints at the time of calling.

* Changes name to IUmbracoEndpointBuilder

* adds note

* Fixing tests, fixing error describers so there's 2x one for back office, one for members, fixes TryConvertTo, fixes login redirect

* fixing build

* Updates user manager to correctly validate password hashing and injects the IBackOfficeUserPasswordChecker

* Merges PR

* Fixes up build and notes

* Implements security stamp and email confirmed for members, cleans up a bunch of repo/service level member groups stuff, shares user store code between members and users and fixes the user identity object so we arent' tracking both groups and roles.

* Security stamp for members is now working

* Fixes keepalive, fixes PublicAccessMiddleware to not throw, updates startup code to be more clear and removes magic that registers middleware.

* adds note

* removes unused filter, fixes build

* fixes WebPath and tests

* Looks up entities in one query

* remove usings

* Fix test, remove stylesheet

* Set status code before we write to response to avoid error

* Ensures that users and members are validated when logging in. Shares more code between users and members.

* merge changes

* oops

* Reducing and removing published member cache

* Fixes RepositoryCacheKeys to ensure the keys are normalized

* oops didn't mean to commit this

* Fix casing issues with caching, stop boxing value types for all cache operations, stop re-creating string keys in DefaultRepositoryCachePolicy

* oops didn't mean to comit this

* bah, far out this keeps getting recommitted. sorry

* cannot inject IPublishedMemberCache and cannot have IPublishedMember

* splits out files, fixes build

* fix tests

* removes membership provider classes

* removes membership provider classes

* updates the identity map definition

* reverts commented out lines

* reverts commented out lines

* Implements members Password config in db, fixes members cookie auth to not interfere with the back office cookie auth, fixes Startup sequence, fixes startup pipeline

* commits change to Startup

* Rename migration from `MemberTableColumns2` to `AddPasswordConfigToMemberTable`

* Fix test

* Fix tests, but adding default passwordConfig to members

Co-authored-by: Bjarke Berg <mail@bergmania.dk>
2021-04-22 15:59:13 +02:00
Shannon Deminick
3792cafb9f Published members cleanup (#10159)
* Getting new netcore PublicAccessChecker in place

* Adds full test coverage for PublicAccessChecker

* remove PublicAccessComposer

* adjust namespaces, ensure RoleManager works, separate public access controller, reduce content controller

* Implements the required methods on IMemberManager, removes old migrated code

* Updates routing to be able to re-route, Fixes middleware ordering ensuring endpoints are last, refactors pipeline options, adds public access middleware, ensures public access follows all hops

* adds note

* adds note

* Cleans up ext methods, ensures that members identity is added on both front-end and back ends. updates how UmbracoApplicationBuilder works in that it explicitly starts endpoints at the time of calling.

* Changes name to IUmbracoEndpointBuilder

* adds note

* Fixing tests, fixing error describers so there's 2x one for back office, one for members, fixes TryConvertTo, fixes login redirect

* fixing build

* Updates user manager to correctly validate password hashing and injects the IBackOfficeUserPasswordChecker

* Merges PR

* Fixes up build and notes

* Implements security stamp and email confirmed for members, cleans up a bunch of repo/service level member groups stuff, shares user store code between members and users and fixes the user identity object so we arent' tracking both groups and roles.

* Security stamp for members is now working

* Fixes keepalive, fixes PublicAccessMiddleware to not throw, updates startup code to be more clear and removes magic that registers middleware.

* adds note

* removes unused filter, fixes build

* fixes WebPath and tests

* Looks up entities in one query

* remove usings

* Fix test, remove stylesheet

* Set status code before we write to response to avoid error

* Ensures that users and members are validated when logging in. Shares more code between users and members.

* merge changes

* oops

* Reducing and removing published member cache

* Fixes RepositoryCacheKeys to ensure the keys are normalized

* oops didn't mean to commit this

* Fix casing issues with caching, stop boxing value types for all cache operations, stop re-creating string keys in DefaultRepositoryCachePolicy

* oops didn't mean to comit this

* bah, far out this keeps getting recommitted. sorry

* cannot inject IPublishedMemberCache and cannot have IPublishedMember

* splits out files, fixes build

* fix tests

* removes membership provider classes

* removes membership provider classes

* updates the identity map definition

* reverts commented out lines

* reverts commented out lines

Co-authored-by: Bjarke Berg <mail@bergmania.dk>
2021-04-22 13:21:43 +02:00