* Handle culture variance and add more options for indexing fields
* Workaround for missing DB on install
* Add a document per language to the delivery API content index
* Filters and Selectors must be able to match multiple values (as OR)
* A few review changes
* Change TODO for a note
* Workaround for lazily resolved field value types in Examine
* Make removal tracking more granular to cover both single and all culture deletion
---------
Co-authored-by: Elitsa <elm@umbraco.dk>
* Make implementations sealed and internal if possible
* Remove usings
* Making some models sealed as well
* Making more models sealed
* Last model changes
* Un-seal ApiContentResponse
---------
Co-authored-by: kjac <kja@umbraco.dk>
* Update the delivery API content index at un/publish
* Make the query service use "itemId" instead of "id" when extracting GUIDs + only fetch that single field rather than all fields
* Simplify the code a little
* Making AncestorsSelectorIndexer sealed
---------
Co-authored-by: Elitsa <elm@umbraco.dk>
* Filter out disallowed content types from the delivery api content index
* Added .
* Allow children of disallowed content types in the index
---------
Co-authored-by: Elitsa <elm@umbraco.dk>
* Introduce content index handlers for the Delivery API
* Do not add protected content to the index
* Re-introduce public collection builder extensions
* Add backwards compatible constructor to ConfigureIndexOptions
* Fix edge case for published children of unpublished parents
* Reduce number of hardcoded field names
* Fixed copy/paste comment
* Fix field definition type for date
* Do not add trashed items to the content index
* Don't expose protected content through the item endpoints
* Removing ReSharper comment
---------
Co-authored-by: Elitsa <elm@umbraco.dk>
* Add the core parts of the headless PoC
* Add Content API project (WIP - loads of TODOs and dupes that need to be fixed!)
* Rename the content API project and namespaces
* Fixed bad merge
* Rename everything "Headless" to "ContentApi" or "Api"
* Refactor Content + Media: Key => Id, Name not nullable
* Make Content API property return value types independent of datatype configuration
* Clean up refactorings
* First stab at an expansion strategy using content picker as example implementation
* Use named JSON options for content API serialization
* Proper inclusion and registration of the content API
* Introduce API media builder
* Make MNTP return API content/media depending on configuration (instead of links) and support output expansion
* Content API: Get by controllers (#13740)
* Adding ContentApiControllerBase
* Adding get by id and url controllers
* Change route of get all test controller
* Rename to ContentApiController
* Refactoring
* Removing test controller
* Content API: Add start-node header value to deal with url collisions (#13746)
* Use start-node header value to deal with url collisions
* Cleanup
* Rename "url" param to "path"
* Adding a start node service to get the start-node header value
* Trim '/' from both beginning and end
* Content API: Support Accept-Language header (#13831)
* Move the content API JSON type resolver to an appropriate namespace
* Add localization based on Accept-Language header
* Content API: Output expansion (#13848)
* Implement request based output expansion strategy + expansion output cache at property level
* Slighty leaner implementation for default output expansion strategy
* Clarify the code a bit
* Fix bad merge
* Encapsulate content API dependencies in the DI
* Support multi-site and multi-culture routing + a little rename/refactor (#13882)
* Support multi-site and multi-culture routing + a little rename/refactor
* Make the by route controller handle root nodes
* Rename Url to Path in API content output
* Add a few comments for magic route creation
* Rename services from "Default" to "Noop"
* Ensure that Umbraco can boot without adding "AddContentApi()" to ConfigureServices
* Moved incorrectly placed media builder
* Fix API routes (#13915)
* Fix multi URL picker value converter trying to access disposed objects in edge cases
* Delivery API: Content routing and structure (#13984)
* Introduce content route instead of content path, rename and rework start item (previously start node) handling
* Strip out start node path in generated route path
* Make the start-item header take precedence over the request domain
* Conditionally enabling the delivery API + add protection and preview support + refactor all services to be singletons + ensure no-op implementations for all required services (#13992)
* Include umbraco properties (width, height, ...) in the Media Properties collection (#14023)
* Move umbraco properties (width, height, ...) to the Properties collection of the API Media model
* Don't output the umbracoFile property of media items
* Add content type deny list (#14025)
* Create a deny list of content types and utilize it for output generation
* Add unit tests
* Dedicated property cache level for Content API (#14027)
* Support redirect tracking (#14033)
* Create a deny list of content types and utilize it for output generation
* Add unit tests
* Handle redirect tracking in the content API
* Include start item routing info for redirects
* Add cultures and their routes to the API output (#14038)
* Create a deny list of content types and utilize it for output generation
* Add unit tests
* Handle redirect tracking in the content API
* Include start item routing info for redirects
* Add culture routes to root output (for HREFLANG support)
* Rename redirect service method to better match its purpose
* Review changes
* Delivery API: Query controller (#14041)
* Initial commit
* Custom ContentAPIFieldDefinitionCollection
* Make index IUmbracoContentIndex
* Add querying for children by parent id (key)
* Add missing interface
* Adding querying endpoint
* Test code
* Compose unpublishedValueSet, so that you get the correct data in the ContentAPI index
* Renaming
* Fix ancestorKeys index values
* Adding IApiQueryExtensionService to be able to query the ContentAPI index in a generic way
* Fix IApiQueryService and clean up QueryContentApiController using it
* Support querying for path
* Fix content API indexing
* Fix default sorting
* Implement concrete QueryOption implementations
* Introduce new ExecuteQuery that uses the Core OptionHandlers
* Implement ExecuteQuery
* Change ExecuteQuery signature and implementation
* Implement demo sorting and fetching
* Add query option handlers and collection builder for them
* Cleanup
* Revert "Conditionally enabling the delivery API + add protection and preview support + refactor all services to be singletons + ensure no-op implementations for all required services (#13992)"
This reverts commit 78e1f748e55383baecd123d06457111e18f13365.
* Revert "Delivery API: Content routing and structure (#13984)"
This reverts commit a0292ae5350362dd6c1c5bc9763deda928c78a75.
* Revert "Fix multi URL picker value converter trying to access disposed objects in edge cases"
This reverts commit 6b7c37a5bf7871bee93a2b2640bbc6ef591f14db.
* Revert "Conditionally enabling the delivery API + add protection and preview support + refactor all services to be singletons + ensure no-op implementations for all required services (#13992)"
This reverts commit 78e1f748e55383baecd123d06457111e18f13365.
* Revert "Delivery API: Content routing and structure (#13984)"
This reverts commit a0292ae5350362dd6c1c5bc9763deda928c78a75.
* Revert "Fix multi URL picker value converter trying to access disposed objects in edge cases"
This reverts commit 6b7c37a5bf7871bee93a2b2640bbc6ef591f14db.
* Fix multi URL picker value converter trying to access disposed objects in edge cases
* Delivery API: Content routing and structure (#13984)
* Introduce content route instead of content path, rename and rework start item (previously start node) handling
* Strip out start node path in generated route path
* Make the start-item header take precedence over the request domain
* Conditionally enabling the delivery API + add protection and preview support + refactor all services to be singletons + ensure no-op implementations for all required services (#13992)
* Test commit
* Refactored interfaces for the query handlers and for the selectors (that will handle the value of the fetch query option)
* Implemented a base class for the query options
* Refactored the names of the selectors and made use of the base class
* Refactored the ApiQueryService
* Refactored the QueryContentApiController.cs
* Conditionally enabling the delivery API + add protection and preview support + refactor all services to be singletons + ensure no-op implementations for all required services (#13992)
* Fixing merge gone wrong
* Fix multi URL picker value converter trying to access disposed objects in edge cases
* Delivery API: Content routing and structure (#13984)
* Introduce content route instead of content path, rename and rework start item (previously start node) handling
* Strip out start node path in generated route path
* Make the start-item header take precedence over the request domain
* Conditionally enabling the delivery API + add protection and preview support + refactor all services to be singletons + ensure no-op implementations for all required services (#13992)
* Make fetching work with the new setup
* Moving files to dedicated folders
* Removing ? for array
* Rename selector query method
* Implement FilterHandler and some filters
* Implement SortHandler and sort some sorts
* Refactoring
* Adding more fields to index due to querying
* Appending filtering and sorting queries
* Implementing a new ISelectorHandler without Examine types
* Re-implementing the collection to have a dedicated one for the selectors
* Implementing a new IFilterHandler without Examine types & refactoring the filters implementing it
* Adding a new collection dedicated to filters
* Renaming the old collection
* Implementing a new ISortHandler without Examine types & refactoring the sorts implementing it
* Adding a new collection for the sorts & adding all collections to UmbracoBuilder.Collections
* Refactoring the service to use the new collections and types
* Refactoring the fields in ContentApiFieldDefinitionCollection
* Remove nullability in Handlers
* Don't return null for selector
* Add TODO for having the filters support negation
* Changing the SortType to FieldType with our custom types on the SortOption
* Fix AncestorsSelector
* Fix ApiQueryService
* Documentation
* Fix Swagger docs
* Refactor the QueryContentApiController
* Adding handling for the IApiContentResponse in the JsonTypeResolver
* Refactor the service to use a safe fallback value in Examine queries
* Adding Noop for the IApiQueryService
* Cleanup
* Remove comment
* Fix name field for indexing
* Don't inherit QueryOptionBase in filters
* Fix casing for API index constant + swap FIXME with TODO
* Add TODO for handling missing fetch with start-item header
* Rename query handler parameters to not leak source (i.e. query string)
---------
Co-authored-by: kjac <kja@umbraco.dk>
Co-authored-by: Elitsa <>
Co-authored-by: Zeegaan <nge@umbraco.dk>
* Delivery API: Adding pagination to query endpoint (#14083)
* Adding pagination to query endpoint
* Optimize the paging using Examine directly
* Fix comment
* Remove skip/take code duplication
---------
Co-authored-by: kjac <kja@umbraco.dk>
* Add missing CompatibilitySuppressions.xml
* Make Delivery API packable
* Make Api.Common packable
* Renamed extension method and namespace so it is discoverable
* Untangle ApiVersion configuration into api.common, so delivery api do not require the management api to boot.
* configure options in management api
* RTE output as JSON for Content API (#14067)
* Conditionally serve RTE output as JSON instead of HTML
* Fixed merge
* Rename to Delivery API (#14119)
* Rename ContentApi to DeliveryApi
* Rename delivery API index implementation
* Update comments from "Content API" to "Delivery API"
* Rename project from Content to Delivery
* Add dedicated controller base for content delivery API
* Rename delivery API content index to include "content" specifically
* Fix compat suppressions
---------
Co-authored-by: kjac <kja@umbraco.dk>
Co-authored-by: Elitsa Marinovska <21998037+elit0451@users.noreply.github.com>
Co-authored-by: Zeegaan <nge@umbraco.dk>
* This fixes issues 11383.
The introduction of the new Examine library caused an unintended consequence that it stopped indexing of nodes with language variants on them when one of the variants was unpublished.
These changes align ValueSetValidationStatus.Filtered to indicate that a node is intended as filtered out of a search, not that parts of its contents had been excluded from the result.
This brings it inline with how it is used in Umbraco.Examine.Lucene/UmbracoContentIndex
Unit tests changed to indicate the intent of ValueSetValidationStatus.Filtered
Change to UmbracoViewPage to make model variable nullable (because the solution wouldn't build otherwise on 2022)
* revert to use explicit type instead of var
Co-authored-by: Nathan Woulfe <nathan@nathanw.com.au>
Co-authored-by: Bjarke Berg <mail@bergmania.dk>
* Don't allocate new object array instances when indexing
There's no reason to allocate new object[] array instances for each ValueSet created for the "y" or "n" values going into the index. This just creates a single object[] for each "y" or "n" value which will save on a ton of allocations when re indexing a bunch of content.
There's an easy way to make further allocation reductions too (prob a separate PR) since there is no reason to create new object[] allocations for: CreatorId, WriterId, Level, TemplateId, Culture. Each of these can be returned from a concurrent dictionary since these values are finite and there won't be too many of each so we can just keep an internal dictionary of these object[] instances.
* fixes mistake
* Updates NoValue and YesValue field declarations - was blowing up
Co-authored-by: Nathan Woulfe <nathan@nathanw.com.au>
* Updating to Examine 3 beta. Currently not building due to missing support for updating values in the OnTransformingIndexValues event
* Updated to beta 6 - It builds, but we need to fix tests
* Fixed issue with values being passed into nested lists.
* Fixed issue in test
* Updated to examine beta.9
* Fixed breaking changes
* Fixed issue with Paths-integers being converted using local culture.
* Align with the old implementation
* Use int.TryParse insteaad of TryConvertTo when we do not want culture specific parsing
* More fixes for cultures and fixed wrong test. Users should be part of all groups to have access
* Fix casing for requested file
* Force tests to not use NLS
* try force tests to not use NLS
* try force tests to not use NLS
* Force tests on windows to run ICU
* More fixes for invariant int parsing
* Change key on actions/emptyRecycleBin, so the casing aligns with the view file, that is named emptyrecyclebin.html
* Fixed casing issue
* use Attempt to align with other code
Currently when the recycle bin is empty, it is going to individually delete each item from the index. This is going to cause tons of allocations in Umbraco for DeferedDeleteIndex objects for each item and then down within Examine is going to process each one individually instead of just doing it in bulk. There will be a lot of allocations made there too along with a bunch of extra and unecessary threads.
* Updates to latest examine, ensures indexes are unlocked on startup, adds more info to the diagnostics and allows for the new sync directory factory to work.
* Gets index syncing working correct and fixes reindexing to not overprocess indexes
* fix duplicate package ref
* rebuilds empty indexes and fixes config with enum
* missing file
* Init commit for examine 2.0 work, most old umb examine tests working, probably a lot that doesn't
* Gets Umbraco Examine tests passing and makes some sense out of them, fixes some underlying issues.
* Large refactor, remove TaskHelper, rename Notifications to be consistent, Gets all examine/lucene indexes building and startup ordered in the correct way, removes old files, creates new IUmbracoIndexingHandler for abstracting out all index operations for umbraco data, abstracts out IIndexRebuilder, Fixes Stack overflow with LiveModelsProvider and loading assemblies, ports some changes from v8 for startup handling with cold boots, refactors out LastSyncedFileManager
* fix up issues with rebuilding and management dashboard.
* removes old files, removes NetworkHelper, fixes LastSyncedFileManager implementation to ensure the machine name is used, fix up logging with cold boot state.
* Makes MainDom safer to use and makes PublishedSnapshotService lazily register with MainDom
* lazily acquire application id (fix unit tests)
* Fixes resource casing and missing test file
* Ensures caches when requiring internal services for PublishedSnapshotService, UseNuCache is a separate call, shouldn't be buried in AddWebComponents, was also causing issues in integration tests since nucache was being used for the Id2Key service.
* For UmbracoTestServerTestBase enable nucache services
* Fixing tests
* Fix another test
* Fixes tests, use TestHostingEnvironment, make Tests.Common use net5, remove old Lucene.Net.Contrib ref.
* Fixes up some review notes
* Fixes issue with doubly registering PublishedSnapshotService meanig there could be 2x instances of it
* Checks for parseexception when executing the query
* Use application root instead of duplicating functionality.
* Added Examine project to netcore only solution file
* Fixed casing issue with LazyLoad, that is not lowercase.
* uses cancellationToken instead of bool flag, fixes always reading lastId from the LastSyncedFileManager, fixes RecurringHostedServiceBase so that there isn't an overlapping thread for the same task type
* Fix tests
* remove legacy test project from solution file
* Fix test
Co-authored-by: Bjarke Berg <mail@bergmania.dk>
* Getting new netcore PublicAccessChecker in place
* Adds full test coverage for PublicAccessChecker
* remove PublicAccessComposer
* adjust namespaces, ensure RoleManager works, separate public access controller, reduce content controller
* Implements the required methods on IMemberManager, removes old migrated code
* Updates routing to be able to re-route, Fixes middleware ordering ensuring endpoints are last, refactors pipeline options, adds public access middleware, ensures public access follows all hops
* adds note
* adds note
* Cleans up ext methods, ensures that members identity is added on both front-end and back ends. updates how UmbracoApplicationBuilder works in that it explicitly starts endpoints at the time of calling.
* Changes name to IUmbracoEndpointBuilder
* adds note
* Fixing tests, fixing error describers so there's 2x one for back office, one for members, fixes TryConvertTo, fixes login redirect
* fixing build
* Updates user manager to correctly validate password hashing and injects the IBackOfficeUserPasswordChecker
* Merges PR
* Fixes up build and notes
* Implements security stamp and email confirmed for members, cleans up a bunch of repo/service level member groups stuff, shares user store code between members and users and fixes the user identity object so we arent' tracking both groups and roles.
* Security stamp for members is now working
* Fixes keepalive, fixes PublicAccessMiddleware to not throw, updates startup code to be more clear and removes magic that registers middleware.
* adds note
* removes unused filter, fixes build
* fixes WebPath and tests
* Looks up entities in one query
* remove usings
* Fix test, remove stylesheet
* Set status code before we write to response to avoid error
* Ensures that users and members are validated when logging in. Shares more code between users and members.
* merge changes
* oops
* Reducing and removing published member cache
* Fixes RepositoryCacheKeys to ensure the keys are normalized
* oops didn't mean to commit this
* Fix casing issues with caching, stop boxing value types for all cache operations, stop re-creating string keys in DefaultRepositoryCachePolicy
* oops didn't mean to comit this
* bah, far out this keeps getting recommitted. sorry
* cannot inject IPublishedMemberCache and cannot have IPublishedMember
* splits out files, fixes build
* fix tests
* removes membership provider classes
* removes membership provider classes
* updates the identity map definition
* reverts commented out lines
* reverts commented out lines
Co-authored-by: Bjarke Berg <mail@bergmania.dk>
