* Working import/export media/document types
* WIP
* Refactoring of import doctype/media types
- added analyze endpoint to extract relevant data without fully processing the file
- split up import endpoints into POST & PUT
- removed availableAtAction as the new endpoint allows clients to call the POST/PUT endpoints with confidence
- Added a new service that is responsible for turning temp files into Import compatible XML and being able to extracty partial information from it
* Wrap persistance access in scopes
* Typos, formatting, clean-up
* PR feedback
* update openapi spec
* Changed deleteFile flag to _temporaryFileService.EnlistDeleteIfScopeCompletes
* Itty bitty typo
* Moved magic cleanup into its own method so orchestration can decide when.
---------
Co-authored-by: Sven Geusens <sge@umbraco.dk>
Co-authored-by: kjac <kja@umbraco.dk>
* Add configuration/code to not run certain tests based on variables/release builds
* Applied longrunning testAttribute to the worst offenders (>200ms on my machine)
* Fix yaml notation
* split up windows/non windows test runs
* Added supression for moved tests
* Fix yaml validation issues
* Change yaml string parameter null value to empty string
* Convert empty strings to whitespace strings
* Rename and cleanup some paramater to better reflect why we use them
* Nightly build test
* Change nightly build authentication type
* template paramater fix
* Update nightly pipeline name
---------
Co-authored-by: Sven Geusens <sge@umbraco.dk>
* Add alias to document item response
* Add master template key to detailed model
* Add mater template key as optiona parameter to Scaffolding
* Check for duplicate alias when creating templates directly
* Clean
* Ensure integration tests creates templates with unique aliases
* Perform mapping in presentation factory
* Throw from IUserIdKeyResolver if id/key not found. This is too critical to not throw.
* Fixed tests
* Explicitly test that we can resolve super user key/ID from their counterparts
---------
Co-authored-by: kjac <kja@umbraco.dk>
* Add default super user key to migrations
* Start refactoring all interfaces signatures with ids
* Refactor datatype service to use userKey pattern instead
* Refactor ContentEditingService to use userkeys
* Refactor services to userKey
* Refactor more services to use userkey instead of id
* Refactor RelationService to use userKeys
* Refactor template service to use keys instead of ids
* Refactor fileservice to use keys instead of ids
* Refactor LocalizationService to use keys instead of ids
* Refactor PackagingService to use keys instead of ids
* Refactor TemplateController to use current user keys
* Refactor DataTypeContainerService.cs
* Refactor DataTypeService to use keys instead of ids
* Fix up tests
* Fix up media editing service to use userkey instead of ID
* Update service ctor to avoid ambigious ctors
* refactor DataTypeService
* Refactor DataTypeService to not have a default value for parentKey
* Apply suggestions from code review
Co-authored-by: Elitsa Marinovska <21998037+elit0451@users.noreply.github.com>
* Update comment
* Add suppression file
* Add backoffice CompatibilitySuppressions
---------
Co-authored-by: Zeegaan <nge@umbraco.dk>
Co-authored-by: Elitsa Marinovska <21998037+elit0451@users.noreply.github.com>
* Use language ISO code for language fallback instead of language ID
* Remove language and language ID from dictionary item and dictionary item translation
* ADd unit test for dictionary item translation value extension
* Make the internal service implementations sealed
* Rename translation ISO code to be more explicit in its origin (Language)
* Add breaking changes suppression
* Handle save of invalid fallback iso code
* Fixed test
* Only allow non-UserCustomCulture's
* Fixed and added tests
* Rename ISO code validation method
* Fix language telemetry test (create Swedish with the correct ISO code)
---------
Co-authored-by: Bjarke Berg <mail@bergmania.dk>
* Align the template services and API with the dictionary ones (use attempt pattern)
* A little controller clean-up
* Mimic old file service behavior, make unit tests happy
* Align CreateForContentTypeAsync return value with the rest of the TemplateService API
* Scaffold endpoint should no longer feature master templates
* Update the OpenAPI JSON
* Rework language service and API
* Revert unintended commit of Directory.Build.props
* Create OS conditional test for invalid ISO codes
* Reintroduce and obsolete old Delete method on ILocalizationService + make new Delete method delete by ISO code + add obsoletion attrs to service implementation
* Review comments + utilize new Delete method
* Do not allow model reuse when creating a new language
* Fix bad merge
* Split localization service into dedicated services for language and dictionary item handling
* Replaced ILocalizationService usage in management API (as much as can be done for now)
* Ensure we can create dictionary items with explicit keys (but no duplicates)
* Fix culture controller so it works properly with pagination
* Update OpenAPI JSON
* Actually update the language being updated...
* Unit test for invalid ISO now no longer needs to differ between OS :)
* A little bit of code health improvements
* A litte less code duplication
* Remove duplicate validation
* Update language models to get and set manual name
* Save custom language name in controller
* Rewrite AngularJS language edit view and controller
* Cleanup language overview
* Remove icon from language overview
* Make styling of control group the same as properties
* Ensure both ISO code and culture name are set in language model
* Use new language model constructor
* Update tests to use new language constructor
* Update culture name in dictionary package export
* Use language name in dictionary
* Fix language nullability issues
* Cleanup GetAllCultures and added null checks
* Re-add obsolete constructors
* Make language name required and update Cypress test
* Fix routing/saveNewLanguages Cypress test
* Make language name optional (improved backwards compatibility)
Co-authored-by: Ronald Barendse <ronald@panoramastudios.nl>
* Created Persistence.SQLite project skeleton.
* SQLite database initialization
* Various changes and hacks to make things work.
* WIP integration tests
* Fix thread safety tests
* Fix tests that relied on tie breaker sorting.
Spent a fair amount of time looking for a less lazy fix but gave up.
* Convert right join to left join ContentTypeRepository.PerformGetByQuery
SQLite doesn't support right join
* Fix test Can_Generate_Delete_SubQuery_Statement
Worth noting that NPoco.DatabaseTypes.SQLiteDatabaseType doesn't override
EscapeSqlIdentifier so NPoco will escape with [].
SQLite docs say > "A keyword enclosed in square brackets is an identifier.
This is not standard SQL.
This quoting mechanism is used by MS Access and SQL Server and is
included in SQLite for compatibility."
Also could have updated SqliteSyntaxProvider to match npoco but
decided against it.
* Fixes for paginated custom order by
* Fix tests broken by lack of unique indexes.
* Fix SqlServerTableByTableTest tests.
These tests didn't actually do anything as the tables already exist so schema creator just returned.
Did however point out that the default implementation for DoesTableExist just returns false so added a default naive implementation.
* Fix ValidateLoginSession - SelectTop must come later
* dry up database cleanup
* Fix up db migration tests.
We can't drop pk in sqlite without recreating table.
Test looks to be testing that add column works as intended which we can test.
* Prevent schema creation errors.
* SQLite ignore lock tests, WAL back on.
* Fix package schema tests
* Fix NPocoFetchTests - case sensitivity not under test
* Fix AdvancedMigrationTests (where possible)
Migrations probably need a good look later.
Maybe nuke old migrations and only support moving to v10 from v9.
If we do that can do some cleanup.
* Cleanup test database configuration
* Run integration tests against SQLite on build agent.
* Drop MS.Data.SQLite
System.Data.SQLite was quicker to roll out due to more CLR type mapping
* YAML
* Skip Umbraco.Tests.Integration.SqlCe
* Drop SqlServerTableByTable tests.
Until this week they did nothing anyway as they with NewSchemaPerTest
so the tests all passed as CreateTable was no op (already exists).
Also all of the tables are created in an empty database by SchemaValidationTest.cs
DatabaseSchemaCreation_Produces_DatabaseSchemaResult_With_Zero_Errors
* Might aswell run against macOS also.
* Copy azure pipelines task header layout
* Delete SQLCe projects
* Remove SQL CE specific code.
* Remove SQL CE NuSpec, template params, build script setup
* Delete umbraco-netcore-only.sln
* Add SkipTests solution configuration and use for codeql
* Remove reference to deleted nuspec file.
* Refactor ConnectionStrings WRT DataDirectory placeholder & ProviderName.
At this point you can try out SQLite support by setting the following
in appsettings.json and then completing the install process.
"ConnectionStrings": {
"umbracoDbDSN": "Data Source=|DataDirectory|/umbraco.sqlite",
"umbracoDbDSN_ProviderName": "System.Data.SQLite"
},
Not currently possible via installer UI without provider name pre-set in
configuration.
* Switch to Microsoft.Data.Sqlite
Some gross hacks but will be good to find out if this works
with apple silicon.
* Enable selection of SQLite via installer UI (also quick install)
* Remove SqlServerDbProviderFactoryCreator to cleanup a TODO
* Move SQL Server support to its own class library
* Add persistence dependencies to Umbraco.CMS metapackage
* Bugfix packages delete query
Created invalid query for SQLite.
* Try out cypress tests Linux + SQLite
* Prevent cypress test artifact upload failure on attempt 2+
* LocalDb bugfixes
* Drop redundant enum
* Move SqlClient constant
* Misc whitespace
* Remove IsSqlCe extension (TODO: drop non 9->10 migrations later).
* Umbraco.Persistence.* -> Umbraco.Cms.Persistence.*
* Display quick install defaults and per provider default database name.
* Misc remove old comment
* little re-arrange
* Remove almost all usages of IsSqlite extension.
* visual adjustments
* Custom Database Configuration is last step and should then say Install.
* use text instead of disabled inputs
* move legend, rename to Install
* Update SqlMainDomLock to work without distributed locks.
* Added IDistributedLockingMechanism interface and in memory impl.
* Drop locking from ISqlSyntaxProvider & wire up scope to abstraction.
* Added SqlServerDistributedLockingMechanism
* Move distributed locking interfaces and exceptions to Core + xmldocs.
* Fix tests, Misc cleanup, Add SQL distributed locking integration tests
* Provide mechanism to specify DistributedLockingMechanism in config
(even if added by composer)
* Nomplementation -> NoImplementation
* Fix misleading comment
* Integration tests use SqlServerDistributedLockingMechanism when possible
* Handle up-gradable locks SqlServerDistributedLockingMechanism.
TODO: InMemoryDistributedLockingMechanism.
Note: Nuked SqlServerDistributedLockingMechanismTests, will still sleep
at night.
Is covered by Umbraco.Cms.Tests.Integration.Umbraco.Infrastructure.Persistence.LockTests
* Make tests pass for InMemoryDistributedLockingMechanism, pretty hacky.
* Tweak constraints on WithCollectionBuilder so i can drop bad constructor
* Added SqliteDistributedLockingMechanism
* Dropped InMemoryDistributedMechanism + magic
InMemoryDistributedMechanism was pretty rubbish and now we have
a decent implementation for SQLite as we no longer block readers
see 8d1f42b.
Also drop the CollectionBuilder setup, instead do the same as we do
for syntax providers etc, it's more automagical so we never require an
explicit selection although we are allowing for it.
However keeping the optional IUmbracoBuilder constructor param for
CollectionBuilders as it's extremely useful.
* Fix quick install "" database name.
* Hide Database Configuration section when a connection string is pre-set.
Doesn't seem worth it to extract db name from connection string.
* Ensure wal test 2+
* Fix logging inconsistencies.
* Ensure in transaction when obtaining locks + no-op the SQLite read lock.
There's no point in running the query just to make a single test pass.
* Fix installer database display names
* Allow SQLite shared cache without losing deferred transactions
* Opt into shared cache for new SQLite databases + fix filename
* Fix misc inconsistency in .gitignore
* Prefer our interceptor interface
* Restore DEBUG_DATABASES code OnConnectionOpened in case it's used.
* Back to private cache.
* Added retry strategy for SQLite + refactor out SQL server specific stuff
* Fix SQL server tests.
* Misc - Orphaned comment, incorrect casing.
* InMemory SQLite test database & turn shared cache back on everywhere.
Co-authored-by: Niels Lyngsø <niels.lyngso@gmail.com>
* Move package to database
* Added migration and implemented new repository
* Updated migrations to use proper xml convert
* Fixed save function and renamed createDate to update date
* Updated dependencyInjection
* Updated UmbracoPlan.cs
* Apply suggestions from code review
Co-authored-by: Elitsa Marinovska <21998037+elit0451@users.noreply.github.com>
* Added File check
* Tried using same context as create table
* Fix DTO
* Fix GetById and local package saving
* Fix images when migrating
* Implement deletion of all local files
* Only delete local repo file, not file snapshots
* Remove static package path and use the one we save
* Update package repo to export package and remove check for ids when exporting
* Minor fixes
* Fix so that you can download package after creating
* Update savePackage method to export package afterwards
Co-authored-by: Nikolaj Geisle <niko737@edu.ucl.dk>
Co-authored-by: Elitsa Marinovska <elm@umbraco.dk>
Co-authored-by: Elitsa Marinovska <21998037+elit0451@users.noreply.github.com>
* cherry-pick 13a51d32 (V8 History cleanup import/export)
Support import/export for doc type history cleanup policy (#11660)
* Support import/export for doc type history cleanup policy
* Support unset/null history cleanup value
* Resolve issue when api endpoints called without cleanup policy.
noop isn't good enough as map fails for response.
* null conditional vs null coalesce assignment
* Don't overwrite existing policy if omitted in import XML
* Update history cleanup warning and translations
* Change history cleanup alert to infomational styling
* Remove margin around history cleanup config
Co-authored-by: Ronald Barendse <ronald@barend.se>
# Conflicts:
# src/Umbraco.Core/Models/IContentType.cs
# src/Umbraco.Core/Packaging/PackageDataInstallation.cs
# src/Umbraco.Infrastructure/Persistence/Repositories/Implement/ContentTypeRepository.cs
# src/Umbraco.Infrastructure/Services/Implement/EntityXmlSerializer.cs
# src/Umbraco.Tests/Packaging/PackageDataInstallationTests.cs
# src/Umbraco.Tests/Services/Importing/ImportResources.Designer.cs
# src/Umbraco.Tests/Umbraco.Tests.csproj
# src/Umbraco.Web.UI/umbraco/config/lang/en.xml
# src/Umbraco.Web.UI/umbraco/config/lang/en_us.xml
# src/Umbraco.Web/Models/Mapping/ContentTypeMapDefinition.cs
# tests/Umbraco.Tests.Integration/Umbraco.Infrastructure/Services/Importing/ImportResources.resx
# tests/Umbraco.Tests.UnitTests/Umbraco.Core/Services/Importing/SingleDocType-WithCleanupPolicy.xml
* Remove namespace aliases
* Update IContentTypeWithHistoryCleanup documentation
Co-authored-by: Ronald Barendse <ronald@barend.se>
