* Pass notification state to cache refreshers.
Pass previous user name into member saved notification state and use when refreshing cache to clear the member by keys based on this.
* Fixed issue raised in code review.
* Fixed casing for state key.
* Added removed parameter to unit tests.
* Fix breaking change.
* v16 cherry pick of member partial cache invalidator see #19314
# Resolved merge conflic in src/Umbraco.Core/Cache/Refreshers/Implement/MemberCacheRefresher.cs
* Take nullmember cacheitems into account
* fix: add appcache null check
* Moved constant into standard location.
Removed now unnecessary comment.
---------
Co-authored-by: Andy Butland <abutland73@gmail.com>
* Clear elementscache from cache refreshers
* Add very simple test ensuring the elements cache is cleared
---------
Co-authored-by: Kenn Jacobsen <kja@umbraco.dk>
* Introduce IsDocumentPublishedInAnyCulture
Sometimes we don't care about culture
* Check ancestor path when resolving cache items
* Fix tests
* Rebuild NavigationService
* Only set node if it has a published ancestor path
* Remove branch when unpublished
* Add tests
* Add seed test
* Consider published ancestor path when seeding documents
* Introduce MediaBreadthFirstKeyProviderTests
This is needed since the logic of document and media is no longer the same
* Remove unused services
* Move assert page to helper
* Add variant tests
* Add tests
* Filter keys in ContentTypeSeedKeyProvider
* Fix tests
* Add failing test showing refreshing issue
* Don't blow up if we can't resolve the node from navigation cache
Turns out that this can actually happen :D Should be fine to just return false
* Refactor cache refresher check
* Make NavigationQueryService service protected
* Add comment on how to refactor breadth first key provider
* Refactor if statement
* Add CacheNullValues option to RepositoryCachePolicy
* Cache null values in DictionaryByKeyRepository
* Fixed issue with nullable reference.
* Updated logic for caching of null values.
* Update src/Umbraco.Infrastructure/Cache/DefaultRepositoryCachePolicy.cs
Co-authored-by: Sven Geusens <geusens@gmail.com>
* Made the NullValueRepresentation overwritable in a generic manner
* Improve generic NullValueCachePolicyResolver
* Revert Commits and clarify logic with comment
This reverts commit 8befb437921cb6e3b87725cefb92a6afbf3d28fb "Improve generic NullValueCachePolicyResolver"
Also reverts 8adf0a2 - Made the NullValueRepresentation overwritable in a generic manner
And 8adf0a2 - Made the NullValueRepresentation overwritable in a generic manner
* Update src/Umbraco.Infrastructure/Cache/DefaultRepositoryCachePolicy.cs
---------
Co-authored-by: Andy Butland <abutland73@gmail.com>
Co-authored-by: Sven Geusens <geusens@gmail.com>
Co-authored-by: Sven Geusens <sge@umbraco.dk>
* Adding contentType to navigation node
* Loading contentType from DB
* Considering contentTypeKey when adding a navigation node & fixing references
* Using IContentTypeBaseService to load content types
* Adding generics to ContentNavigationServiceBase and fixing references
* Adding TryGetChildrenKeysOfType and implementation
* Refactoring test data
* Adding unit tests for TryGetChildrenKeysOfType
* Update CreateContentCreateModel in tests to receive content type
* Fix references
* Cleanup
* Adding integration tests for TryGetChildrenKeysOfType
* Cleanup
* Cleanup
* Descendants of type implementation
* Descendants of type tests
* Interface updates
* Ancestors of type implementation and tests
* Siblings of type implementation and tests
* Cleanup
* Integration tests
* Adding root of type implementation and tests
* Fix Ancestors extension methods
* Fix descendants extension methods
* Fix children extension methods
* Fix siblings extension methods
* Add helper methods
* Fix a bug
* Fixed unit tests by setting up mocks
* Adding missing extension method
---------
Co-authored-by: Bjarke Berg <mail@bergmania.dk>
* Check for id later.
* Return 501 status codes from obsoleted endpoints
* Use contants
* Ensure ordering of cache refresher is not changes
---------
Co-authored-by: Bjarke Berg <mail@bergmania.dk>
* Replace obsolete UserGroup Alias consts to key equivalent in tests
* Update use of usergroup alias consts to key equivalent in IsSystemgroup extension method
* Obsolete (internally) unused helper function which purpose doesn't even seem true
* Prepped EmbedProviders for proper removal of non async methods and unneeded proxy methods
* Remove obsoleted UmbracoPath and updated internal references
* Corrected mistake and updated unittets
* Update usergroup tests that use aliases for "system" groups
* Replace more uses of globalsettings.UmbracoPath
* Remove GetDateType by key non async
* Cleanup some usages of hostingEnvironment.MapPathContentRoot
* More easy obsoletion cleanup
* Small Typeload cleanup
* More obsolete removal
* Deploy obsoletion cleanup
* Remove obsolete methods from OEmbedProviderBase.cs
---------
Co-authored-by: Zeegaan <skrivdetud@gmail.com>
* Move DocumentCacheService
* Add clear all documentws from memory cache
* Fix RedirectTracker
* Implement refresh node/branch/all/delete
* Only update databasecache in RefreshContentAsync
* Fix tests
* Skip blueprints in cache
* Clear caches when contenttype is updated
* Clear cache on data type update
* Refresh media
* Only update memory cache from refreshers
* Fix imports
* Add named options
* Use cache entry settings in media
* Obsolete nucache settings
---------
Co-authored-by: Bjarke Berg <mail@bergmania.dk>
* bump version to 15.1.0
* V15 Fixed the failing smoke tests in the pipeline v15 (#17158)
* Fixed the failing tests of Member Group due to UI changes
* Fixed the failing tests of Member due to UI changes
* Fixed the failing tests of User due to UI changes
* Fixed failing tests for Dictionary and Document Type
* Updated tests due to test helper changes
* Bumped version
* Updated assert steps due to the response changes
* Updated tests due to api helper changes
* Updated tests due to UI changes
* Fixed tests for delete partial view
* Fixed tests
* Added more waits
* Updated assert steps
* Fixed failing tests for Block Grid and Media
* Added more waits
* Added skip tests
* Removed waits time
* Updated assertion steps for User
* Added todo
* Updated tests due to api helper changes
* Bumped version
* Added skip tests
* Fetch sortOrder for each navigationNode
* Update NavigationNode to have sortOrder and change Parent and Children props to keys instead of NavigationNodes
* Consider sortOrder when building the navigation structures
* Renaming tests
* Adding tests for items being the last in structure when added, moved, etc.
* Updating names
* Cleanup
* Updating cache refreshers with changes due to sorting
* Refactoring due to sorting changes and resolving key to NavigationNode
* Removing sortOrder params from test as they are calculated automatically
* Adding content and media integration tests to test sorting functionality
* Adding sortOrder param for special case when adding new nodes
* Adding new UpdateSortOrder to INavigationManagementService
* Revert "V15 Fixed the failing smoke tests in the pipeline v15 (#17158)"
This reverts commit 31399c3b15.
* Revert "bump version to 15.1.0"
This reverts commit 5e4d15be
* Fix revert
* Add sort order when creating media
---------
Co-authored-by: Jacob Overgaard <752371+iOvergaard@users.noreply.github.com>
Co-authored-by: Nhu Dinh <150406148+nhudinh0309@users.noreply.github.com>
Co-authored-by: Bjarke Berg <mail@bergmania.dk>
* Ensure routing respect publish status
* Check published status per culture
* Added PublishStatusService to get publish status for a given documentkey and culture
* Added tests and fixed bug with a static fields that should not have been static
* Make sure the write and read cache key is always the same no matter where the request comes from
There is an edge case where the incomming culure is fully capitalized while the read is camelcase
* Fixed review comments
---------
Co-authored-by: Sven Geusens <sge@umbraco.dk>
* Remove nucache reference from Web.Common
* Get tests building-ish
* Move ReservedFieldNamesService to the right project
* Remove IPublishedSnapshotStatus
* Added functionality to the INavigationQueryService to get root keys
* Fixed issue with navigation
* Remove IPublishedSnapshot from UmbracoContext
* Begin removing usage of IPublishedSnapshot from PublishedContentExtensions
* Fix PublishedContentExtensions.cs
* Don't use snapshots in delivery media api
* Use IPublishedMediaCache in QueryMediaApiController
* Remove more usages of IPublishedSnapshotAccessor
* Comment out tests
* Remove more usages of PublishedSnapshotAccessor
* Remove PublishedSnapshot from property
* Fixed test build
* Fix errors
* Fix some tests
* Delete NuCache 🎉
* Implement DatabaseCacheRebuilder
* Remove usage of IPublishedSnapshotService
* Remove IPublishedSnapshotService
* Remove TestPublishedSnapshotAccessor and make tests build
* Don't test Snapshot cachelevel
It's no longer supported
* Fix BlockEditorConverter
Element != Element document type
* Remember to set cachemanager
* Fix RichTextParserTests
* Implement TryGetLevel on INavigationQueryService
* Fake level and obsolete it in PublishedContent
* Remove ChildrenForAllCultures
* Hack Path property on PublishedContent
* Remove usages of IPublishedSnapshot in tests
* More ConvertersTests
* Add hybrid cache to integration tests
We can actually do this now because we no longer save files on disk
* Rename IPublishedSnapshotRebuilder to ICacheRebuilder
* Comment out tests
* V15: Replacing the usages of Parent (navigation data) from IPublishedContent (#17125)
* Fix .Parent references in PublishedContentExtensions
* Add missing methods to FriendlyPublishedContentExtensions (ones that you were able to call on the content directly as they now require extra params)
* Fix references from the extension methods
* Fix dependencies in tests
* Replace IPublishedSnapshotAccessor with the content cache in tests
* Resolving more .Parent references
* Fix unit tests
* Obsolete and use extension methods
* Remove private method and use extension instead
* Moving code around
* Fix tests
* Fix more references
* Cleanup
* Fix more usages
* Resolve merge conflict
* Fix tests
* Cleanup
* Fix more tests
* Fixed unit tests
* Cleanup
* Replace last usages
---------
Co-authored-by: Bjarke Berg <mail@bergmania.dk>
* Remove usage of IPublishedSnapshotAccessor from IRequestItemProvider
* Post merge fixup
* Remo IPublishedSnapshot
* Add HasAny to IDocumentUrlService
* Fix TextBuilder
* Fix modelsbuilder tests
* Use explicit types
* Implement GetByContentType
* Support element types in PublishedContentTypeCache
* Run enlistments before publishing notifications
* Fix elements cache refreshing
* Implement GetByUdi
* Implement GetAtRoot
* Implement GetByRoute
* Reimplement GetRouteById
* Fix blocks unit tests
* Initialize domain cache on boot
* Only return routes with domains on non default lanauges
* V15: Replacing the usages of `Children` (navigation data) from `IPublishedContent` (#17159)
* Update params in PublishedContentExtensions to the general interfaces for the published cache and navigation service, so that we can use the extension methods on both documents and media
* Introduce GetParent() which uses the right services
* Fix obsolete message on .Parent
* Obsolete .Children
* Fix usages of Children for ApiMediaQueryService
* Fix usage in internal
* Fix usages in views
* Fix indentation
* Fix issue with delete language
* Update nuget pacakges
* Clear elements cache when content is deleted
instead of trying to update it
* Reset publishedModelFactory
* Fixed publishing
---------
Co-authored-by: Bjarke Berg <mail@bergmania.dk>
Co-authored-by: Elitsa Marinovska <21998037+elit0451@users.noreply.github.com>
Co-authored-by: kjac <kja@umbraco.dk>
* Handle navigation updates in cache refeshers
* Same for media cache refreshers
* Clean up
* More clean up and renaming content to media
* Update src/Umbraco.Core/Services/Navigation/ContentNavigationServiceBase.cs
Co-authored-by: Elitsa Marinovska <21998037+elit0451@users.noreply.github.com>
---------
Co-authored-by: Elitsa <elm@umbraco.dk>
Co-authored-by: Elitsa Marinovska <21998037+elit0451@users.noreply.github.com>
* Started work on service
* temp work
* temp commit
* Temp commit
* Added more routing logic
* Fixed tests
* Refactor and prepare for isdraft
* Work on drafts
* Fixed tests
* Move to enlistment to ensure caches is only updated on scope complete
* Clean up and handle null cultures
* Added functionality to the INavigationQueryService to get root keys
* Added migration
* Fixed issue with navigation
* Added migration
* Temp commit, move to cache refreshers.
* Fixed issues
* List urls
* fix build
* Fixed integration tests
* Refactor to create new content finder instead of changing the old
* rollback wrong commited line
* Clean up, and use docuemnt url service for index
* Fixed List endpoin
* Do not use Navigation service in methods intended by management api
* Fixed examine tests
* Make methods virtual
* Use domain from published request
* Use hybrid cache from new content finder
* Eliminate nucache usage
* Fixed issue with delivery api and url generation
* Fixed linux tests
* Added hybrid cache to all integration tests
* Update to dotnet 9 and update nuget packages
* Update umbraco code version
* Update Directory.Build.props
Co-authored-by: Elitsa Marinovska <21998037+elit0451@users.noreply.github.com>
* Include preview version in pipeline
* update template projects
* update global json with specific version
* Update version.json to v15
* Rename TrimStart and TrimEnd to string specific
* Rename to Exact
* Update global.json
Co-authored-by: Ronald Barendse <ronald@barend.se>
* Remove includePreviewVersion
* Rename to trim exact
* Add new Hybridcache project
* Add tests
* Start implementing PublishedContent.cs
* Implement repository for content
* Refactor to use async everywhere
* Add cache refresher
* make public as needed for serialization
* Use content type cache to get content type out
* Refactor to use ContentCacheNode model, that goes in the memory cache
* Remove content node kit as its not needed
* Implement tests for ensuring caching
* Implement better asserts
* Implement published property
* Refactor to use mapping
* Rename to document tests
* Update to test properties
* Create more tests
* Refactor mock tests into own file
* Update property test
* Fix published version of content
* Change default cache level to elements
* Refactor to always have draft
* Refactor to not use PublishedModelFactory
* Added tests
* Added and updated tests
* Fixed tests
* Don't return empty object with id
* More tests
* Added key
* Another key
* Refactor CacheService to be responsible for using the hybrid cache
* Use notification handler to remove deleted content from cache
* Add more tests for missing functions
* Implement missing methods
* Remove HasContent as it pertains to routing
* Fik up test
* formatting
* refactor variable names
* Implement variant tests
* Map all the published content properties
* Get item out of cache first, to assert updated
* Implement member cache
* Add member test
* Implement media cache
* Implement property tests for media tests
* Refactor tests to use extension method
* Add more media tests
* Refactor properties to no longer have element caching
* Don't use property cache level
* Start implementing seeding
* Only seed when main
* Add Immutable for performance
* Implement permanent seeding of content
* Implement cache settings
* Implement tests for seeding
* Update package version
* start refactoring nurepo
* Refactor so draft & published nodes are cached individually
* Refactor RefreshContent to take node instead of IContent
* Refactor media to also use cache nodes
* Remove member from repo as it isn't cached
* Refactor media to not include preview, as media has no draft
* create new benchmark project
* POC Integration benchmarks with custom api controllers
* Start implementing content picker tests
* Implement domain cache
* Rework content cache to implement interface
* Start implementing elements cache
* Implement published snapshot service
* Publish snapshot tests
* Use snapshot for elements cache
* Create test proving we don't clear cache when updating content picker
* Clear entire elements cache
* Remove properties from element cache, when content gets updated.
* Rename methods to async
* Refactor to use old cache interfaces instead of new ones
* Remove snapshot, as it is no longer needed
* Fix tests building
* Refactor domaincache to not have snapshots
* Delete benchmarks
* Delete benchmarks
* Add HybridCacheProject to Umbraco
* Add comment to route value transformer
* Implement is draft
* remove snapshot from property
* V15 updated the hybrid caching integration tests to use ContentEditingService (#16947)
* Added builder extension withParentKey
* Created builder with ContentEditingService
* Added usage of the ContentEditingService to SETUP
* Started using ContentEditingService builder in tests
* Updated builder extensions
* Fixed builder
* Clean up
* Clean up, not done
* Added Ids
* Remove entries from cache on delete
* Fix up seeding logic
* Don't register hybrid cache twice
* Change seeded entry options
* Update hybrid cache package
* Fix up published property to work with delivery api again
* Fix dependency injection to work with tests
* Fix naming
* Dont make caches nullable
* Make content node sealed
* Remove path and other unused from content node
* Remove hacky 2 phase ctor
* Refactor to actually set content templates
* Remove umbraco context
* Remove "HasBy" methods
* rename property data
* Delete obsolete legacy stuff
* Add todo for making expiration configurable
* Add todo in UmbracoContext
* Add clarifying comment in content factory
* Remove xml stuff from published property
* Fix according to review
* Make content type cache injectible
* Make content type cache injectible
* Rename to database cache repository
* Rename to document cache
* Add TODO
* Refactor to async
* Rename to async
* Make everything async
* Remove duplicate line from json schema
* Move Hybrid cache project
* Remove leftover file
* Refactor to use keys
* Refactor published content to no longer have content data, as it is on the node itself
* Refactor to member to use proper content node ctor
* Move tests to own folder
* Add immutable objects to property and content data for performance
* Make property data public
* Fix member caching to be singleton
* Obsolete GetContentType
* Remove todo
* Fix naming
* Fix lots of exposed errors due to scope test
* Add final scope tests
* Rename to document cache service
* Rename test files
* Create new doc type tests
* Add ignore to tests
* Start implementing refresh for content type save
* Clear contenttype cache when contenttype is updated
* Fix test
Teh contenttype is not upated unless the property is dirty
* Use init for ContentSourceDto
* Fix get by key in PublishedContentTypeCache
* Remove ContentType from PublishedContentTypeCache when contenttype is deleted
* Update to preview 7
* Fix versions
* Increase timeout for sqlite integration tests
* Undo timeout increase
* Try and undo init change to ContentSourceDto
* That wasn't it chief
* Try and make DomainAndUrlsTests non NonParallelizable
* Update versions
* Only run cache tests on linux for now
---------
Co-authored-by: Elitsa Marinovska <21998037+elit0451@users.noreply.github.com>
Co-authored-by: Ronald Barendse <ronald@barend.se>
Co-authored-by: Andreas Zerbst <andr317c@live.dk>
Co-authored-by: Sven Geusens <sge@umbraco.dk>
Co-authored-by: Andreas Zerbst <73799582+andr317c@users.noreply.github.com>
Co-authored-by: nikolajlauridsen <nikolajlauridsen@protonmail.ch>
* Cache user by id
* Add todo
* Fix tests
* Clear cache after update
* Refer to base class cache instead of implementing own
* Dont hardcode cache key
* Update to be Payload cache refresher instead
* Implement using keymap for member
* Remove current usages of GetUserById
* User userId resolver to resolve user key
* Refactor user repository to use GUID not int
* Add happy path test
* Remove user in cache when user gets updated
* Use await in async method
* Fix up according to review
* Update IMetricsConsentService.cs to have async method
* Fix according to review
* Fix more according to comments
* Revert "Fix up according to review"
This reverts commit a75acaaa
* Get current backoffice user from method
* Update user repository delete functionality
* Fix up more test
* Try to get user by id if key fails
* Add user key as required claim
* Fix tests
* Don't set claim in BackofficeController
* Create constant for the Sub claim
---------
Co-authored-by: kjac <kja@umbraco.dk>
* Added missing locks, and ensure we have timeout on all of them
* Wrap the removable of objects from the hashset in a lock
* remove old comment
* Exit correct lock
* Added ExecuteNonQuery(DbCommand command) on database to ensure we call OnExecutingCommand and OnExecutedCommand when executing DbCommands
* Added Cache Instructions lock, to avoid deadlocks
* Optimized read locks for nucache when only one content type is rebuilt
* Optimized the SqlServer locks, so only one command is executed (and thereby roundtrip) per lock instead of two
* Avoid breaking changes
* Cosmetic changes
* Take locks if everything is rebuild
* Use same lock in scopes, to avoid potential deadlocks between the two
* Use eager locks in PublishedSnapshotService.cs
* Added timeouts to some of the application locks
* Revert "Use eager locks in PublishedSnapshotService.cs"
This reverts commit 01873aae978ffa6e6686d253e482c493715e3a96.
* Revert "Added Cache Instructions lock, to avoid deadlocks"
This reverts commit e3fca7c12a804bb32ca1156b8abd42a957e9dc21.
* Use single readlock call to lock many
* Use eager locks for reads
* Eager write locks
* Ignore test of lazy locks
* Unique timeout exception messages
---------
Co-authored-by: kjac <kja@umbraco.dk>
* Migrate ObjectJsonExtensions
* Use more generic exception to not use Newtonsoft
It should matter if it's a JsonReaderException, if we can't read we can't read
* Remove obsoleted constructors
* Use more generic exception in ContentValueSetBuilder
* Fix constructors
* Remove UdiRangeJsonConverter
* Remove more legacy newtonsoft stuff
* Migrate away from newtonsoft in CacheInstructionService
* Remove unused model binders
* Remove more newtonsoft
* Remove newtonsoft from DatabaseServerMessenger
* Remove now irrelevant benchmark
* Remove the usage of Newtonsoft from ImageCropperTemplateCoreExtensions
The value converter will never return JObject, JsonDocument, or JsonNode
* Remove usages of newtonsoft in ComplexPropertyEditorContentNotificationHandler
JTokens are no longer returned, so we don't need to check for it
* Remove newtonsoft references
* Re-add newtonsoft dependency to Umbraco.Tests.Common
* Fix package references
* move dependency
---------
Co-authored-by: Bjarke Berg <mail@bergmania.dk>
* Introduced IDataTypeConfigurationCache
* Applied IDataTypeConfigurationCache to Property Editors and display mapping
* Invalidate new cache trough DataTypeConfigurationCacheRefresher
* Improve IDatatype service to use cached int path when fetching by guid (using idkeymap)
---------
Co-authored-by: Sven Geusens <sge@umbraco.dk>
Co-authored-by: Ronald Barendse <ronald@barend.se>
* Bugfix: MediaCacheRefresher needs to always clear the mediaCache no matter what the publishedState is
* fix: check correct permissions for deleteDocumentFromRecycleBin
* Fix: ImageCropper propertyValues should not hold invalid values.
* Added media delete endpoints
* PR comment fix: Do not schedule cleanup if we know the file does not exist.
* resolved forward merge build conflicts
namespace cleanup
---------
Co-authored-by: Sven Geusens <sge@umbraco.dk>
