2021-05-18 19:07:17 +02:00
name : $(TeamProject)_$(Build.DefinitionName)_$(SourceBranchName)_$(Date:yyyyMMdd)$(Rev:.r)
v10 SQLite support + distributed locking abstractions (#11922)
* Created Persistence.SQLite project skeleton.
* SQLite database initialization
* Various changes and hacks to make things work.
* WIP integration tests
* Fix thread safety tests
* Fix tests that relied on tie breaker sorting.
Spent a fair amount of time looking for a less lazy fix but gave up.
* Convert right join to left join ContentTypeRepository.PerformGetByQuery
SQLite doesn't support right join
* Fix test Can_Generate_Delete_SubQuery_Statement
Worth noting that NPoco.DatabaseTypes.SQLiteDatabaseType doesn't override
EscapeSqlIdentifier so NPoco will escape with [].
SQLite docs say > "A keyword enclosed in square brackets is an identifier.
This is not standard SQL.
This quoting mechanism is used by MS Access and SQL Server and is
included in SQLite for compatibility."
Also could have updated SqliteSyntaxProvider to match npoco but
decided against it.
* Fixes for paginated custom order by
* Fix tests broken by lack of unique indexes.
* Fix SqlServerTableByTableTest tests.
These tests didn't actually do anything as the tables already exist so schema creator just returned.
Did however point out that the default implementation for DoesTableExist just returns false so added a default naive implementation.
* Fix ValidateLoginSession - SelectTop must come later
* dry up database cleanup
* Fix up db migration tests.
We can't drop pk in sqlite without recreating table.
Test looks to be testing that add column works as intended which we can test.
* Prevent schema creation errors.
* SQLite ignore lock tests, WAL back on.
* Fix package schema tests
* Fix NPocoFetchTests - case sensitivity not under test
* Fix AdvancedMigrationTests (where possible)
Migrations probably need a good look later.
Maybe nuke old migrations and only support moving to v10 from v9.
If we do that can do some cleanup.
* Cleanup test database configuration
* Run integration tests against SQLite on build agent.
* Drop MS.Data.SQLite
System.Data.SQLite was quicker to roll out due to more CLR type mapping
* YAML
* Skip Umbraco.Tests.Integration.SqlCe
* Drop SqlServerTableByTable tests.
Until this week they did nothing anyway as they with NewSchemaPerTest
so the tests all passed as CreateTable was no op (already exists).
Also all of the tables are created in an empty database by SchemaValidationTest.cs
DatabaseSchemaCreation_Produces_DatabaseSchemaResult_With_Zero_Errors
* Might aswell run against macOS also.
* Copy azure pipelines task header layout
* Delete SQLCe projects
* Remove SQL CE specific code.
* Remove SQL CE NuSpec, template params, build script setup
* Delete umbraco-netcore-only.sln
* Add SkipTests solution configuration and use for codeql
* Remove reference to deleted nuspec file.
* Refactor ConnectionStrings WRT DataDirectory placeholder & ProviderName.
At this point you can try out SQLite support by setting the following
in appsettings.json and then completing the install process.
"ConnectionStrings": {
"umbracoDbDSN": "Data Source=|DataDirectory|/umbraco.sqlite",
"umbracoDbDSN_ProviderName": "System.Data.SQLite"
},
Not currently possible via installer UI without provider name pre-set in
configuration.
* Switch to Microsoft.Data.Sqlite
Some gross hacks but will be good to find out if this works
with apple silicon.
* Enable selection of SQLite via installer UI (also quick install)
* Remove SqlServerDbProviderFactoryCreator to cleanup a TODO
* Move SQL Server support to its own class library
* Add persistence dependencies to Umbraco.CMS metapackage
* Bugfix packages delete query
Created invalid query for SQLite.
* Try out cypress tests Linux + SQLite
* Prevent cypress test artifact upload failure on attempt 2+
* LocalDb bugfixes
* Drop redundant enum
* Move SqlClient constant
* Misc whitespace
* Remove IsSqlCe extension (TODO: drop non 9->10 migrations later).
* Umbraco.Persistence.* -> Umbraco.Cms.Persistence.*
* Display quick install defaults and per provider default database name.
* Misc remove old comment
* little re-arrange
* Remove almost all usages of IsSqlite extension.
* visual adjustments
* Custom Database Configuration is last step and should then say Install.
* use text instead of disabled inputs
* move legend, rename to Install
* Update SqlMainDomLock to work without distributed locks.
* Added IDistributedLockingMechanism interface and in memory impl.
* Drop locking from ISqlSyntaxProvider & wire up scope to abstraction.
* Added SqlServerDistributedLockingMechanism
* Move distributed locking interfaces and exceptions to Core + xmldocs.
* Fix tests, Misc cleanup, Add SQL distributed locking integration tests
* Provide mechanism to specify DistributedLockingMechanism in config
(even if added by composer)
* Nomplementation -> NoImplementation
* Fix misleading comment
* Integration tests use SqlServerDistributedLockingMechanism when possible
* Handle up-gradable locks SqlServerDistributedLockingMechanism.
TODO: InMemoryDistributedLockingMechanism.
Note: Nuked SqlServerDistributedLockingMechanismTests, will still sleep
at night.
Is covered by Umbraco.Cms.Tests.Integration.Umbraco.Infrastructure.Persistence.LockTests
* Make tests pass for InMemoryDistributedLockingMechanism, pretty hacky.
* Tweak constraints on WithCollectionBuilder so i can drop bad constructor
* Added SqliteDistributedLockingMechanism
* Dropped InMemoryDistributedMechanism + magic
InMemoryDistributedMechanism was pretty rubbish and now we have
a decent implementation for SQLite as we no longer block readers
see 8d1f42b.
Also drop the CollectionBuilder setup, instead do the same as we do
for syntax providers etc, it's more automagical so we never require an
explicit selection although we are allowing for it.
However keeping the optional IUmbracoBuilder constructor param for
CollectionBuilders as it's extremely useful.
* Fix quick install "" database name.
* Hide Database Configuration section when a connection string is pre-set.
Doesn't seem worth it to extract db name from connection string.
* Ensure wal test 2+
* Fix logging inconsistencies.
* Ensure in transaction when obtaining locks + no-op the SQLite read lock.
There's no point in running the query just to make a single test pass.
* Fix installer database display names
* Allow SQLite shared cache without losing deferred transactions
* Opt into shared cache for new SQLite databases + fix filename
* Fix misc inconsistency in .gitignore
* Prefer our interceptor interface
* Restore DEBUG_DATABASES code OnConnectionOpened in case it's used.
* Back to private cache.
* Added retry strategy for SQLite + refactor out SQL server specific stuff
* Fix SQL server tests.
* Misc - Orphaned comment, incorrect casing.
* InMemory SQLite test database & turn shared cache back on everywhere.
Co-authored-by: Niels Lyngsø <niels.lyngso@gmail.com>
2022-03-11 16:14:20 +00:00
2022-05-19 10:25:44 +01:00
parameters :
- name : sqlServerIntegrationTests
displayName : Run SQL Server Integration Tests
type : boolean
default : false
2024-11-20 12:19:46 +01:00
- name : sqlServerLinuxAcceptanceTests
displayName : Run SQL Server Linux Acceptance Tests
type : boolean
default : false
2022-05-19 10:25:44 +01:00
- name : myGetDeploy
displayName : Deploy to MyGet
type : boolean
default : false
- name : nuGetDeploy
displayName : Deploy to NuGet
type : boolean
default : false
2023-12-05 10:54:23 +01:00
- name : npmDeploy
displayName : Deploy to Npm
type : boolean
default : false
2022-05-19 10:25:44 +01:00
- name : buildApiDocs
displayName : Build API docs
type : boolean
default : false
- name : uploadApiDocs
displayName : Upload API docs
type : boolean
default : false
2023-09-12 14:16:27 +02:00
- name : forceReleaseTestFilter
displayName : Force to use the release test filters
type : boolean
default : false
- name : integrationNonReleaseTestFilter
displayName : TestFilter used for non-release type builds
type : string
2024-11-09 10:28:23 +01:00
default : "--filter TestCategory!=LongRunning&TestCategory!=NonCritical"
2023-09-12 14:16:27 +02:00
- name : integrationReleaseTestFilter
displayName : TestFilter used for release type builds
type : string
2024-11-09 10:28:23 +01:00
default : " "
2023-09-12 14:16:27 +02:00
- name : nonWindowsIntegrationNonReleaseTestFilter
2024-02-02 10:17:40 +01:00
displayName : TestFilter used for non-release type builds on non Windows agents
2023-09-12 14:16:27 +02:00
type : string
2024-11-09 10:28:23 +01:00
default : "--filter TestCategory!=LongRunning&TestCategory!=NonCritical"
2023-09-12 14:16:27 +02:00
- name : nonWindowsIntegrationReleaseTestFilter
2024-02-02 10:17:40 +01:00
displayName : TestFilter used for release type builds on non Windows agents
2023-09-12 14:16:27 +02:00
type : string
2024-11-09 10:28:23 +01:00
default : " "
2023-12-05 10:54:23 +01:00
- name : isNightly
2024-11-09 10:28:23 +01:00
displayName : "Is nightly build (used for MyGet feed)"
2023-12-05 10:54:23 +01:00
type : boolean
default : false
v10 SQLite support + distributed locking abstractions (#11922)
* Created Persistence.SQLite project skeleton.
* SQLite database initialization
* Various changes and hacks to make things work.
* WIP integration tests
* Fix thread safety tests
* Fix tests that relied on tie breaker sorting.
Spent a fair amount of time looking for a less lazy fix but gave up.
* Convert right join to left join ContentTypeRepository.PerformGetByQuery
SQLite doesn't support right join
* Fix test Can_Generate_Delete_SubQuery_Statement
Worth noting that NPoco.DatabaseTypes.SQLiteDatabaseType doesn't override
EscapeSqlIdentifier so NPoco will escape with [].
SQLite docs say > "A keyword enclosed in square brackets is an identifier.
This is not standard SQL.
This quoting mechanism is used by MS Access and SQL Server and is
included in SQLite for compatibility."
Also could have updated SqliteSyntaxProvider to match npoco but
decided against it.
* Fixes for paginated custom order by
* Fix tests broken by lack of unique indexes.
* Fix SqlServerTableByTableTest tests.
These tests didn't actually do anything as the tables already exist so schema creator just returned.
Did however point out that the default implementation for DoesTableExist just returns false so added a default naive implementation.
* Fix ValidateLoginSession - SelectTop must come later
* dry up database cleanup
* Fix up db migration tests.
We can't drop pk in sqlite without recreating table.
Test looks to be testing that add column works as intended which we can test.
* Prevent schema creation errors.
* SQLite ignore lock tests, WAL back on.
* Fix package schema tests
* Fix NPocoFetchTests - case sensitivity not under test
* Fix AdvancedMigrationTests (where possible)
Migrations probably need a good look later.
Maybe nuke old migrations and only support moving to v10 from v9.
If we do that can do some cleanup.
* Cleanup test database configuration
* Run integration tests against SQLite on build agent.
* Drop MS.Data.SQLite
System.Data.SQLite was quicker to roll out due to more CLR type mapping
* YAML
* Skip Umbraco.Tests.Integration.SqlCe
* Drop SqlServerTableByTable tests.
Until this week they did nothing anyway as they with NewSchemaPerTest
so the tests all passed as CreateTable was no op (already exists).
Also all of the tables are created in an empty database by SchemaValidationTest.cs
DatabaseSchemaCreation_Produces_DatabaseSchemaResult_With_Zero_Errors
* Might aswell run against macOS also.
* Copy azure pipelines task header layout
* Delete SQLCe projects
* Remove SQL CE specific code.
* Remove SQL CE NuSpec, template params, build script setup
* Delete umbraco-netcore-only.sln
* Add SkipTests solution configuration and use for codeql
* Remove reference to deleted nuspec file.
* Refactor ConnectionStrings WRT DataDirectory placeholder & ProviderName.
At this point you can try out SQLite support by setting the following
in appsettings.json and then completing the install process.
"ConnectionStrings": {
"umbracoDbDSN": "Data Source=|DataDirectory|/umbraco.sqlite",
"umbracoDbDSN_ProviderName": "System.Data.SQLite"
},
Not currently possible via installer UI without provider name pre-set in
configuration.
* Switch to Microsoft.Data.Sqlite
Some gross hacks but will be good to find out if this works
with apple silicon.
* Enable selection of SQLite via installer UI (also quick install)
* Remove SqlServerDbProviderFactoryCreator to cleanup a TODO
* Move SQL Server support to its own class library
* Add persistence dependencies to Umbraco.CMS metapackage
* Bugfix packages delete query
Created invalid query for SQLite.
* Try out cypress tests Linux + SQLite
* Prevent cypress test artifact upload failure on attempt 2+
* LocalDb bugfixes
* Drop redundant enum
* Move SqlClient constant
* Misc whitespace
* Remove IsSqlCe extension (TODO: drop non 9->10 migrations later).
* Umbraco.Persistence.* -> Umbraco.Cms.Persistence.*
* Display quick install defaults and per provider default database name.
* Misc remove old comment
* little re-arrange
* Remove almost all usages of IsSqlite extension.
* visual adjustments
* Custom Database Configuration is last step and should then say Install.
* use text instead of disabled inputs
* move legend, rename to Install
* Update SqlMainDomLock to work without distributed locks.
* Added IDistributedLockingMechanism interface and in memory impl.
* Drop locking from ISqlSyntaxProvider & wire up scope to abstraction.
* Added SqlServerDistributedLockingMechanism
* Move distributed locking interfaces and exceptions to Core + xmldocs.
* Fix tests, Misc cleanup, Add SQL distributed locking integration tests
* Provide mechanism to specify DistributedLockingMechanism in config
(even if added by composer)
* Nomplementation -> NoImplementation
* Fix misleading comment
* Integration tests use SqlServerDistributedLockingMechanism when possible
* Handle up-gradable locks SqlServerDistributedLockingMechanism.
TODO: InMemoryDistributedLockingMechanism.
Note: Nuked SqlServerDistributedLockingMechanismTests, will still sleep
at night.
Is covered by Umbraco.Cms.Tests.Integration.Umbraco.Infrastructure.Persistence.LockTests
* Make tests pass for InMemoryDistributedLockingMechanism, pretty hacky.
* Tweak constraints on WithCollectionBuilder so i can drop bad constructor
* Added SqliteDistributedLockingMechanism
* Dropped InMemoryDistributedMechanism + magic
InMemoryDistributedMechanism was pretty rubbish and now we have
a decent implementation for SQLite as we no longer block readers
see 8d1f42b.
Also drop the CollectionBuilder setup, instead do the same as we do
for syntax providers etc, it's more automagical so we never require an
explicit selection although we are allowing for it.
However keeping the optional IUmbracoBuilder constructor param for
CollectionBuilders as it's extremely useful.
* Fix quick install "" database name.
* Hide Database Configuration section when a connection string is pre-set.
Doesn't seem worth it to extract db name from connection string.
* Ensure wal test 2+
* Fix logging inconsistencies.
* Ensure in transaction when obtaining locks + no-op the SQLite read lock.
There's no point in running the query just to make a single test pass.
* Fix installer database display names
* Allow SQLite shared cache without losing deferred transactions
* Opt into shared cache for new SQLite databases + fix filename
* Fix misc inconsistency in .gitignore
* Prefer our interceptor interface
* Restore DEBUG_DATABASES code OnConnectionOpened in case it's used.
* Back to private cache.
* Added retry strategy for SQLite + refactor out SQL server specific stuff
* Fix SQL server tests.
* Misc - Orphaned comment, incorrect casing.
* InMemory SQLite test database & turn shared cache back on everywhere.
Co-authored-by: Niels Lyngsø <niels.lyngso@gmail.com>
2022-03-11 16:14:20 +00:00
2022-05-19 10:25:44 +01:00
variables :
2023-10-24 13:59:42 +02:00
nodeVersion : 20
2022-09-27 14:22:34 +02:00
solution : umbraco.sln
2022-05-19 10:25:44 +01:00
buildConfiguration : Release
2022-09-27 14:22:34 +02:00
UMBRACO__CMS__GLOBAL__ID : 00000000 -0000 -0000 -0000 -000000000042
DOTNET_NOLOGO : true
DOTNET_GENERATE_ASPNET_CERTIFICATE : false
DOTNET_SKIP_FIRST_TIME_EXPERIENCE : true
DOTNET_CLI_TELEMETRY_OPTOUT : true
2024-07-01 12:23:58 +02:00
npm_config_cache : $(Pipeline.Workspace)/.npm_client
NODE_OPTIONS : --max_old_space_size=16384
2021-03-22 14:20:30 +01:00
2022-05-19 10:25:44 +01:00
stages :
###############################################
## Build
###############################################
- stage : Build
jobs :
- job : A
displayName : Build Umbraco CMS
pool :
2024-11-09 10:28:23 +01:00
vmImage : "windows-latest"
2022-05-19 10:25:44 +01:00
steps :
2023-02-16 15:30:52 +01:00
- checkout : self
2024-11-29 12:20:19 +01:00
submodules : false
lfs : false ,
fetchDepth : 500
2024-07-01 12:23:58 +02:00
- task : UseDotNet@2
displayName : Use .NET SDK from global.json
2022-05-19 10:25:44 +01:00
inputs :
2024-07-01 12:23:58 +02:00
useGlobalJson : true
2022-09-27 14:22:34 +02:00
- task : DotNetCoreCLI@2
displayName : Run dotnet restore
inputs :
command : restore
projects : $(solution)
2022-05-19 10:25:44 +01:00
- task : DotNetCoreCLI@2
2023-10-11 11:25:05 +02:00
name : build
displayName : Run dotnet build and generate NuGet packages
2022-05-19 10:25:44 +01:00
inputs :
command : build
2022-09-27 14:22:34 +02:00
projects : $(solution)
2024-11-09 10:28:23 +01:00
arguments : "--configuration $(buildConfiguration) --no-restore --property:ContinuousIntegrationBuild=true --property:GeneratePackageOnBuild=true --property:PackageOutputPath=$(Build.ArtifactStagingDirectory)/nupkg"
2022-05-19 10:25:44 +01:00
- task : PublishPipelineArtifact@1
displayName : Publish nupkg
inputs :
targetPath : $(Build.ArtifactStagingDirectory)/nupkg
artifactName : nupkg
- task : PublishPipelineArtifact@1
displayName : Publish build artifacts
inputs :
targetPath : $(Build.SourcesDirectory)
artifactName : build_output
2024-07-01 12:23:58 +02:00
- job : B
displayName : Build Bellissima Package
pool :
2024-11-09 10:28:23 +01:00
vmImage : "ubuntu-latest"
2024-07-01 12:23:58 +02:00
steps :
- checkout : self
2024-11-29 12:20:19 +01:00
submodules : false
lfs : false ,
fetchDepth : 1
fetchFilter : tree:0
2024-07-01 12:23:58 +02:00
- template : templates/backoffice-install.yml
- script : npm run build:for:npm
displayName : Run build:for:npm
workingDirectory : src/Umbraco.Web.UI.Client
2023-12-05 10:54:23 +01:00
- bash : |
echo "##[command]Running npm pack"
echo "##[debug]Output directory: $(Build.ArtifactStagingDirectory)"
mkdir $(Build.ArtifactStagingDirectory)/npm
npm pack --pack-destination $(Build.ArtifactStagingDirectory)/npm
mv .npmrc $(Build.ArtifactStagingDirectory)/npm/
2024-07-01 12:23:58 +02:00
displayName : Run npm pack
2024-03-12 10:17:42 +01:00
workingDirectory : src/Umbraco.Web.UI.Client
2023-12-05 10:54:23 +01:00
- task : PublishPipelineArtifact@1
displayName : Publish Bellissima npm artifact
inputs :
targetPath : $(Build.ArtifactStagingDirectory)/npm
artifactName : npm
v10 SQLite support + distributed locking abstractions (#11922)
* Created Persistence.SQLite project skeleton.
* SQLite database initialization
* Various changes and hacks to make things work.
* WIP integration tests
* Fix thread safety tests
* Fix tests that relied on tie breaker sorting.
Spent a fair amount of time looking for a less lazy fix but gave up.
* Convert right join to left join ContentTypeRepository.PerformGetByQuery
SQLite doesn't support right join
* Fix test Can_Generate_Delete_SubQuery_Statement
Worth noting that NPoco.DatabaseTypes.SQLiteDatabaseType doesn't override
EscapeSqlIdentifier so NPoco will escape with [].
SQLite docs say > "A keyword enclosed in square brackets is an identifier.
This is not standard SQL.
This quoting mechanism is used by MS Access and SQL Server and is
included in SQLite for compatibility."
Also could have updated SqliteSyntaxProvider to match npoco but
decided against it.
* Fixes for paginated custom order by
* Fix tests broken by lack of unique indexes.
* Fix SqlServerTableByTableTest tests.
These tests didn't actually do anything as the tables already exist so schema creator just returned.
Did however point out that the default implementation for DoesTableExist just returns false so added a default naive implementation.
* Fix ValidateLoginSession - SelectTop must come later
* dry up database cleanup
* Fix up db migration tests.
We can't drop pk in sqlite without recreating table.
Test looks to be testing that add column works as intended which we can test.
* Prevent schema creation errors.
* SQLite ignore lock tests, WAL back on.
* Fix package schema tests
* Fix NPocoFetchTests - case sensitivity not under test
* Fix AdvancedMigrationTests (where possible)
Migrations probably need a good look later.
Maybe nuke old migrations and only support moving to v10 from v9.
If we do that can do some cleanup.
* Cleanup test database configuration
* Run integration tests against SQLite on build agent.
* Drop MS.Data.SQLite
System.Data.SQLite was quicker to roll out due to more CLR type mapping
* YAML
* Skip Umbraco.Tests.Integration.SqlCe
* Drop SqlServerTableByTable tests.
Until this week they did nothing anyway as they with NewSchemaPerTest
so the tests all passed as CreateTable was no op (already exists).
Also all of the tables are created in an empty database by SchemaValidationTest.cs
DatabaseSchemaCreation_Produces_DatabaseSchemaResult_With_Zero_Errors
* Might aswell run against macOS also.
* Copy azure pipelines task header layout
* Delete SQLCe projects
* Remove SQL CE specific code.
* Remove SQL CE NuSpec, template params, build script setup
* Delete umbraco-netcore-only.sln
* Add SkipTests solution configuration and use for codeql
* Remove reference to deleted nuspec file.
* Refactor ConnectionStrings WRT DataDirectory placeholder & ProviderName.
At this point you can try out SQLite support by setting the following
in appsettings.json and then completing the install process.
"ConnectionStrings": {
"umbracoDbDSN": "Data Source=|DataDirectory|/umbraco.sqlite",
"umbracoDbDSN_ProviderName": "System.Data.SQLite"
},
Not currently possible via installer UI without provider name pre-set in
configuration.
* Switch to Microsoft.Data.Sqlite
Some gross hacks but will be good to find out if this works
with apple silicon.
* Enable selection of SQLite via installer UI (also quick install)
* Remove SqlServerDbProviderFactoryCreator to cleanup a TODO
* Move SQL Server support to its own class library
* Add persistence dependencies to Umbraco.CMS metapackage
* Bugfix packages delete query
Created invalid query for SQLite.
* Try out cypress tests Linux + SQLite
* Prevent cypress test artifact upload failure on attempt 2+
* LocalDb bugfixes
* Drop redundant enum
* Move SqlClient constant
* Misc whitespace
* Remove IsSqlCe extension (TODO: drop non 9->10 migrations later).
* Umbraco.Persistence.* -> Umbraco.Cms.Persistence.*
* Display quick install defaults and per provider default database name.
* Misc remove old comment
* little re-arrange
* Remove almost all usages of IsSqlite extension.
* visual adjustments
* Custom Database Configuration is last step and should then say Install.
* use text instead of disabled inputs
* move legend, rename to Install
* Update SqlMainDomLock to work without distributed locks.
* Added IDistributedLockingMechanism interface and in memory impl.
* Drop locking from ISqlSyntaxProvider & wire up scope to abstraction.
* Added SqlServerDistributedLockingMechanism
* Move distributed locking interfaces and exceptions to Core + xmldocs.
* Fix tests, Misc cleanup, Add SQL distributed locking integration tests
* Provide mechanism to specify DistributedLockingMechanism in config
(even if added by composer)
* Nomplementation -> NoImplementation
* Fix misleading comment
* Integration tests use SqlServerDistributedLockingMechanism when possible
* Handle up-gradable locks SqlServerDistributedLockingMechanism.
TODO: InMemoryDistributedLockingMechanism.
Note: Nuked SqlServerDistributedLockingMechanismTests, will still sleep
at night.
Is covered by Umbraco.Cms.Tests.Integration.Umbraco.Infrastructure.Persistence.LockTests
* Make tests pass for InMemoryDistributedLockingMechanism, pretty hacky.
* Tweak constraints on WithCollectionBuilder so i can drop bad constructor
* Added SqliteDistributedLockingMechanism
* Dropped InMemoryDistributedMechanism + magic
InMemoryDistributedMechanism was pretty rubbish and now we have
a decent implementation for SQLite as we no longer block readers
see 8d1f42b.
Also drop the CollectionBuilder setup, instead do the same as we do
for syntax providers etc, it's more automagical so we never require an
explicit selection although we are allowing for it.
However keeping the optional IUmbracoBuilder constructor param for
CollectionBuilders as it's extremely useful.
* Fix quick install "" database name.
* Hide Database Configuration section when a connection string is pre-set.
Doesn't seem worth it to extract db name from connection string.
* Ensure wal test 2+
* Fix logging inconsistencies.
* Ensure in transaction when obtaining locks + no-op the SQLite read lock.
There's no point in running the query just to make a single test pass.
* Fix installer database display names
* Allow SQLite shared cache without losing deferred transactions
* Opt into shared cache for new SQLite databases + fix filename
* Fix misc inconsistency in .gitignore
* Prefer our interceptor interface
* Restore DEBUG_DATABASES code OnConnectionOpened in case it's used.
* Back to private cache.
* Added retry strategy for SQLite + refactor out SQL server specific stuff
* Fix SQL server tests.
* Misc - Orphaned comment, incorrect casing.
* InMemory SQLite test database & turn shared cache back on everywhere.
Co-authored-by: Niels Lyngsø <niels.lyngso@gmail.com>
2022-03-11 16:14:20 +00:00
2022-05-19 10:25:44 +01:00
- stage : Build_Docs
2023-10-18 14:52:08 +02:00
condition : and(succeeded(), or(eq(dependencies.Build.outputs['A.build.NBGV_PublicRelease'], 'True'), ${{parameters.buildApiDocs}}))
2022-05-19 10:25:44 +01:00
displayName : Prepare API Documentation
dependsOn : Build
variables :
2023-10-11 11:25:05 +02:00
umbracoMajorVersion : $[ stageDependencies.Build.A.outputs['build.NBGV_VersionMajor'] ]
2022-05-19 10:25:44 +01:00
jobs :
# C# API Reference
- job :
displayName : Build C# API Reference
pool :
2024-11-09 10:28:23 +01:00
vmImage : "windows-latest"
2022-05-19 10:25:44 +01:00
steps :
2023-10-30 13:55:26 +01:00
- task : UseDotNet@2
2024-02-07 14:31:16 +01:00
displayName : Use .NET SDK from global.json
2023-10-30 13:55:26 +01:00
inputs :
2024-02-07 14:31:16 +01:00
useGlobalJson : true
2022-05-19 10:25:44 +01:00
- task : PowerShell@2
displayName : Install DocFX
inputs :
2022-09-19 16:37:24 +02:00
targetType : inline
script : |
2023-12-05 12:38:14 +01:00
choco install docfx --version=2.59.4 -y
2022-09-19 16:37:24 +02:00
if ($lastexitcode -ne 0){
throw ("Error installing DocFX")
}
2022-05-19 10:25:44 +01:00
- task : PowerShell@2
displayName : Generate metadata
inputs :
2022-09-19 16:37:24 +02:00
targetType : inline
script : |
docfx metadata "$(Build.SourcesDirectory)/build/csharp-docs/docfx.json"
if ($lastexitcode -ne 0){
throw ("Error generating metadata.")
}
2022-05-19 10:25:44 +01:00
- task : PowerShell@2
displayName : Generate documentation
inputs :
2022-09-19 16:37:24 +02:00
targetType : inline
script : |
docfx build "$(Build.SourcesDirectory)/build/csharp-docs/docfx.json"
if ($lastexitcode -ne 0){
throw ("Error generating documentation.")
}
2022-05-19 10:25:44 +01:00
- task : ArchiveFiles@2
displayName : Archive C# Docs
inputs :
2022-09-19 16:37:24 +02:00
rootFolderOrFile : $(Build.SourcesDirectory)/build/csharp-docs/_site
includeRootFolder : false
archiveFile : $(Build.ArtifactStagingDirectory)/csharp-docs.zip
2022-05-19 10:25:44 +01:00
- task : PublishPipelineArtifact@1
displayName : Publish C# Docs
inputs :
2022-09-19 16:37:24 +02:00
targetPath : $(Build.ArtifactStagingDirectory)/csharp-docs.zip
artifact : csharp-docs
2021-03-23 19:28:15 +01:00
2022-05-19 10:25:44 +01:00
# js API Reference
- job :
displayName : Build js API Reference
pool :
2024-11-09 10:28:23 +01:00
vmImage : "ubuntu-latest"
2023-06-12 14:12:02 +02:00
variables :
BASE_PATH : /v$(umbracoMajorVersion)/ui
2022-05-19 10:25:44 +01:00
steps :
2023-06-12 14:12:02 +02:00
- checkout : self
2024-11-29 12:20:19 +01:00
submodules : false
lfs : false ,
fetchDepth : 1
fetchFilter : tree:0
2024-07-01 12:23:58 +02:00
- template : templates/backoffice-install.yml
2023-06-12 14:12:02 +02:00
- script : npm run storybook:build
displayName : Build Storybook
env :
VITE_BASE_PATH : $(BASE_PATH)/
2024-03-12 10:17:42 +01:00
workingDirectory : $(Build.SourcesDirectory)/src/Umbraco.Web.UI.Client
2023-06-12 14:12:02 +02:00
- script : sed -i "s|/umbraco/backoffice|$(BASE_PATH)/umbraco/backoffice|" assets/*.js
displayName : Replace BASE_PATH on assets
2024-03-12 10:17:42 +01:00
workingDirectory : $(Build.SourcesDirectory)/src/Umbraco.Web.UI.Client/storybook-static
2022-05-19 10:25:44 +01:00
- task : ArchiveFiles@2
2024-07-02 09:40:53 +02:00
displayName : Archive Storybook
2022-05-19 10:25:44 +01:00
inputs :
2024-03-12 10:17:42 +01:00
rootFolderOrFile : $(Build.SourcesDirectory)/src/Umbraco.Web.UI.Client/storybook-static
2022-09-19 16:37:24 +02:00
includeRootFolder : false
archiveFile : $(Build.ArtifactStagingDirectory)/ui-docs.zip
2022-05-19 10:25:44 +01:00
- task : PublishPipelineArtifact@1
2024-07-02 09:40:53 +02:00
displayName : Publish Storybook
2022-05-19 10:25:44 +01:00
inputs :
2022-09-19 16:37:24 +02:00
targetPath : $(Build.ArtifactStagingDirectory)/ui-docs.zip
artifact : ui-docs
2024-07-02 09:40:53 +02:00
- script : npm run generate:ui-api-docs
displayName : Generate API Docs
workingDirectory : $(Build.SourcesDirectory)/src/Umbraco.Web.UI.Client
- task : ArchiveFiles@2
displayName : Archive UI API Docs
inputs :
rootFolderOrFile : $(Build.SourcesDirectory)/src/Umbraco.Web.UI.Client/ui-api
includeRootFolder : false
archiveFile : $(Build.ArtifactStagingDirectory)/ui-api-docs.zip
- task : PublishPipelineArtifact@1
displayName : Publish UI API Docs
inputs :
targetPath : $(Build.ArtifactStagingDirectory)/ui-api-docs.zip
artifact : ui-api-docs
2021-05-18 19:07:17 +02:00
2022-05-19 10:25:44 +01:00
###############################################
## Test
###############################################
- stage : Unit
displayName : Unit Tests
dependsOn : Build
jobs :
# Unit Tests
- job :
displayName : Unit Tests
strategy :
matrix :
Windows :
2024-11-09 10:28:23 +01:00
vmImage : "windows-latest"
2022-05-19 10:25:44 +01:00
Linux :
2024-11-09 10:28:23 +01:00
vmImage : "ubuntu-latest"
2022-05-19 10:25:44 +01:00
macOS :
2024-11-09 10:28:23 +01:00
vmImage : "macOS-latest"
2022-05-19 10:25:44 +01:00
pool :
vmImage : $(vmImage)
steps :
2024-11-29 12:20:19 +01:00
- checkout : self
submodules : false
lfs : false ,
fetchDepth : 1
fetchFilter : tree:0
2022-05-19 10:25:44 +01:00
- task : DownloadPipelineArtifact@2
displayName : Download build artifacts
inputs :
artifact : build_output
path : $(Build.SourcesDirectory)
- task : UseDotNet@2
2024-02-02 10:17:40 +01:00
displayName : Use .NET SDK from global.json
2022-05-19 10:25:44 +01:00
inputs :
2024-02-02 10:17:40 +01:00
useGlobalJson : true
2022-05-19 10:25:44 +01:00
- task : DotNetCoreCLI@2
displayName : Run dotnet test
inputs :
command : test
2024-11-09 10:28:23 +01:00
projects : "tests/Umbraco.Tests.UnitTests/Umbraco.Tests.UnitTests.csproj"
arguments : "--configuration $(buildConfiguration) --no-build"
2022-05-19 10:25:44 +01:00
testRunTitle : Unit Tests - $(Agent.OS)
2021-03-18 14:48:22 +01:00
2022-05-19 10:25:44 +01:00
- stage : Integration
displayName : Integration Tests
dependsOn : Build
2023-10-18 14:58:44 +02:00
variables :
releaseTestFilter : eq(dependencies.Build.outputs['A.build.NBGV_PublicRelease'], 'True')
2022-05-19 10:25:44 +01:00
jobs :
# Integration Tests (SQLite)
- job :
2024-10-18 12:09:46 +02:00
timeoutInMinutes : 180
2022-05-19 10:25:44 +01:00
displayName : Integration Tests (SQLite)
strategy :
matrix :
2024-11-09 10:28:23 +01:00
# Windows:
# vmImage: 'windows-latest'
2022-05-19 10:25:44 +01:00
Linux :
2024-11-26 12:07:26 +01:00
vmImage : "ubuntu-24.04"
2022-05-19 10:25:44 +01:00
macOS :
2024-11-09 10:28:23 +01:00
vmImage : "macOS-latest"
2022-05-19 10:25:44 +01:00
pool :
vmImage : $(vmImage)
2024-02-02 10:17:40 +01:00
variables :
2024-11-09 10:28:23 +01:00
Tests__Database__DatabaseType : "Sqlite"
2022-05-19 10:25:44 +01:00
steps :
2024-11-29 12:20:19 +01:00
- checkout : self
submodules : false
lfs : false ,
fetchDepth : 1
fetchFilter : tree:0
2024-02-02 10:17:40 +01:00
# Setup test environment
2022-05-19 10:25:44 +01:00
- task : DownloadPipelineArtifact@2
displayName : Download build artifacts
inputs :
artifact : build_output
path : $(Build.SourcesDirectory)
2024-02-02 10:17:40 +01:00
2022-05-19 10:25:44 +01:00
- task : UseDotNet@2
2024-02-02 10:17:40 +01:00
displayName : Use .NET SDK from global.json
2022-05-19 10:25:44 +01:00
inputs :
2024-02-02 10:17:40 +01:00
useGlobalJson : true
# Test
2022-05-19 10:25:44 +01:00
- task : DotNetCoreCLI@2
2024-02-02 10:17:40 +01:00
displayName : Run dotnet test
2023-09-12 14:16:27 +02:00
inputs :
command : test
2024-11-09 10:28:23 +01:00
projects : "tests/Umbraco.Tests.Integration/Umbraco.Tests.Integration.csproj"
2023-09-12 14:16:27 +02:00
testRunTitle : Integration Tests SQLite - $(Agent.OS)
2024-02-02 10:17:40 +01:00
${{ if and(eq(variables['Agent.OS'],'Windows_NT'), or(variables.releaseTestFilter, parameters.forceReleaseTestFilter)) }}:
2024-11-09 10:28:23 +01:00
arguments : "--configuration $(buildConfiguration) --no-build ${{parameters.integrationReleaseTestFilter}}"
2024-02-02 10:17:40 +01:00
${{ elseif eq(variables['Agent.OS'],'Windows_NT') }}:
2024-11-09 10:28:23 +01:00
arguments : "--configuration $(buildConfiguration) --no-build ${{parameters.integrationNonReleaseTestFilter}}"
2024-02-02 10:17:40 +01:00
${{ elseif or(variables.releaseTestFilter, parameters.forceReleaseTestFilter) }}:
2024-11-09 10:28:23 +01:00
arguments : "--configuration $(buildConfiguration) --no-build ${{parameters.nonWindowsIntegrationReleaseTestFilter}}"
2023-09-12 14:16:27 +02:00
${{ else }}:
2024-11-09 10:28:23 +01:00
arguments : "--configuration $(buildConfiguration) --no-build ${{parameters.nonWindowsIntegrationNonReleaseTestFilter}}"
2021-03-18 14:48:22 +01:00
2022-05-19 10:25:44 +01:00
# Integration Tests (SQL Server)
- job :
2024-01-23 08:49:25 +01:00
timeoutInMinutes : 180
2024-02-22 12:22:28 +01:00
# condition: or(eq(stageDependencies.Build.A.outputs['build.NBGV_PublicRelease'], 'True'), ${{parameters.sqlServerIntegrationTests}}) # Outcommented due to timeouts
condition : eq(${{parameters.sqlServerIntegrationTests}}, True)
2022-05-19 10:25:44 +01:00
displayName : Integration Tests (SQL Server)
strategy :
matrix :
Windows :
2024-11-09 10:28:23 +01:00
vmImage : "windows-latest"
2024-02-02 10:17:40 +01:00
Tests__Database__DatabaseType : LocalDb
Tests__Database__SQLServerMasterConnectionString : N /A
2022-05-19 10:25:44 +01:00
Linux :
2024-11-09 10:28:23 +01:00
vmImage : "ubuntu-latest"
2024-02-02 10:17:40 +01:00
SA_PASSWORD : UmbracoIntegration123!
Tests__Database__DatabaseType : SqlServer
2024-11-09 10:28:23 +01:00
Tests__Database__SQLServerMasterConnectionString : "Server=(local);User Id=sa;Password=$(SA_PASSWORD);TrustServerCertificate=True"
2022-05-19 10:25:44 +01:00
pool :
vmImage : $(vmImage)
steps :
2024-02-02 10:17:40 +01:00
# Setup test environment
2022-05-19 10:25:44 +01:00
- task : DownloadPipelineArtifact@2
displayName : Download build artifacts
inputs :
artifact : build_output
path : $(Build.SourcesDirectory)
2024-02-02 10:17:40 +01:00
2022-10-19 10:46:32 +02:00
- task : UseDotNet@2
2024-02-02 10:17:40 +01:00
displayName : Use .NET SDK from global.json
2022-10-19 10:46:32 +02:00
inputs :
2024-02-02 10:17:40 +01:00
useGlobalJson : true
# Start SQL Server
- powershell : docker run --name mssql -d -p 1433:1433 -e "ACCEPT_EULA=Y" -e "MSSQL_SA_PASSWORD=$(SA_PASSWORD)" mcr.microsoft.com/mssql/server:2022-latest
displayName : Start SQL Server Docker image (Linux)
2022-05-19 10:25:44 +01:00
condition : and(succeeded(), eq(variables['Agent.OS'], 'Linux'))
2024-02-02 10:17:40 +01:00
- pwsh : SqlLocalDB start MSSQLLocalDB
displayName : Start SQL Server LocalDB (Windows)
condition : and(succeeded(), eq(variables['Agent.OS'], 'Windows_NT'))
# Test
2022-05-19 10:25:44 +01:00
- task : DotNetCoreCLI@2
2024-02-02 10:17:40 +01:00
displayName : Run dotnet test
2023-09-12 14:16:27 +02:00
inputs :
command : test
2024-11-09 10:28:23 +01:00
projects : "tests/Umbraco.Tests.Integration/Umbraco.Tests.Integration.csproj"
2023-09-12 14:16:27 +02:00
testRunTitle : Integration Tests SQL Server - $(Agent.OS)
2024-02-02 10:17:40 +01:00
${{ if and(eq(variables['Agent.OS'],'Windows_NT'), or(variables.releaseTestFilter, parameters.forceReleaseTestFilter)) }}:
2024-11-09 10:28:23 +01:00
arguments : "--configuration $(buildConfiguration) --no-build ${{parameters.integrationReleaseTestFilter}}"
2024-02-02 10:17:40 +01:00
${{ elseif eq(variables['Agent.OS'],'Windows_NT') }}:
2024-11-09 10:28:23 +01:00
arguments : "--configuration $(buildConfiguration) --no-build ${{parameters.integrationNonReleaseTestFilter}}"
2024-02-02 10:17:40 +01:00
${{ elseif or(variables.releaseTestFilter, parameters.forceReleaseTestFilter) }}:
2024-11-09 10:28:23 +01:00
arguments : "--configuration $(buildConfiguration) --no-build ${{parameters.nonWindowsIntegrationReleaseTestFilter}}"
2023-09-12 14:16:27 +02:00
${{ else }}:
2024-11-09 10:28:23 +01:00
arguments : "--configuration $(buildConfiguration) --no-build ${{parameters.nonWindowsIntegrationNonReleaseTestFilter}}"
2024-02-02 10:17:40 +01:00
# Stop SQL Server
- pwsh : docker stop mssql
displayName : Stop SQL Server Docker image (Linux)
condition : and(succeeded(), eq(variables['Agent.OS'], 'Linux'))
- pwsh : SqlLocalDB stop MSSQLLocalDB
displayName : Stop SQL Server LocalDB (Windows)
condition : and(succeeded(), eq(variables['Agent.OS'], 'Windows_NT'))
2021-03-18 14:48:22 +01:00
2024-01-12 19:32:25 +07:00
- stage : E2E
displayName : E2E Tests
dependsOn : Build
2024-02-02 10:17:40 +01:00
variables :
npm_config_cache : $(Pipeline.Workspace)/.npm_e2e
# Enable console logging in Release mode
SERILOG__WRITETO__0__NAME : Async
SERILOG__WRITETO__0__ARGS__CONFIGURE__0__NAME : Console
# Set unattended install settings
UMBRACO__CMS__UNATTENDED__INSTALLUNATTENDED : true
UMBRACO__CMS__UNATTENDED__UNATTENDEDUSERNAME : Playwright Test
UMBRACO__CMS__UNATTENDED__UNATTENDEDUSERPASSWORD : UmbracoAcceptance123!
UMBRACO__CMS__UNATTENDED__UNATTENDEDUSEREMAIL : playwright@umbraco.com
# Custom Umbraco settings
UMBRACO__CMS__CONTENT__CONTENTVERSIONCLEANUPPOLICY__ENABLECLEANUP : false
UMBRACO__CMS__GLOBAL__DISABLEELECTIONFORSINGLESERVER : true
UMBRACO__CMS__GLOBAL__INSTALLMISSINGDATABASE : true
UMBRACO__CMS__GLOBAL__ID : 00000000 -0000 -0000 -0000 -000000000042
UMBRACO__CMS__GLOBAL__VERSIONCHECKPERIOD : 0
UMBRACO__CMS__GLOBAL__USEHTTPS : true
UMBRACO__CMS__HEALTHCHECKS__NOTIFICATION__ENABLED : false
UMBRACO__CMS__KEEPALIVE__DISABLEKEEPALIVETASK : true
UMBRACO__CMS__WEBROUTING__UMBRACOAPPLICATIONURL : https://localhost:44331/
ASPNETCORE_URLS : https://localhost:44331
2024-01-12 19:32:25 +07:00
jobs :
# E2E Tests
- job :
2024-02-02 10:17:40 +01:00
displayName : E2E Tests (SQLite)
2024-01-12 19:32:25 +07:00
variables :
2024-02-02 10:17:40 +01:00
# Connection string
CONNECTIONSTRINGS__UMBRACODBDSN : Data Source=Umbraco;Mode=Memory;Cache=Shared;Foreign Keys=True;Pooling=True
CONNECTIONSTRINGS__UMBRACODBDSN_PROVIDERNAME : Microsoft.Data.Sqlite
2024-01-12 19:32:25 +07:00
strategy :
matrix :
2024-12-02 14:48:15 +01:00
LinuxPart1Of3 :
2024-11-09 10:28:23 +01:00
vmImage : "ubuntu-latest"
2024-12-02 14:48:15 +01:00
testCommand : "npx playwright test DefaultConfig --grep \"@smoke\"--shard=1/3"
LinuxPart2Of3 :
vmImage : "ubuntu-latest"
testCommand : "npx playwright test DefaultConfig --grep \"@smoke\" --shard=2/3"
LinuxPart3Of3 :
vmImage : "ubuntu-latest"
testCommand : "npx playwright test DefaultConfig --grep \"@smoke\" --shard=3/3"
WindowsPart1Of3 :
2024-11-09 10:28:23 +01:00
vmImage : "windows-latest"
2024-12-02 14:48:15 +01:00
testCommand : "npx playwright test DefaultConfig --grep \"@smoke\" --shard=1/3"
WindowsPart2Of3 :
vmImage : "windows-latest"
testCommand : "npx playwright test DefaultConfig --grep \"@smoke\" --shard=2/3"
WindowsPart3Of3 :
vmImage : "windows-latest"
testCommand : "npx playwright test DefaultConfig --grep \"@smoke\" --shard=3/3"
2024-01-12 19:32:25 +07:00
pool :
vmImage : $(vmImage)
steps :
2024-02-02 10:17:40 +01:00
# Setup test environment
2024-01-12 19:32:25 +07:00
- task : DownloadPipelineArtifact@2
2024-02-02 10:17:40 +01:00
displayName : Download NuGet artifacts
2024-01-12 19:32:25 +07:00
inputs :
artifact : nupkg
2024-02-02 10:17:40 +01:00
path : $(Agent.BuildDirectory)/app/nupkg
2024-01-12 19:32:25 +07:00
- task : NodeTool@0
displayName : Use Node.js $(nodeVersion)
retryCountOnTaskFailure : 3
inputs :
versionSpec : $(nodeVersion)
2024-02-02 10:17:40 +01:00
- task : UseDotNet@2
displayName : Use .NET SDK from global.json
inputs :
useGlobalJson : true
- pwsh : |
"UMBRACO_USER_LOGIN=$(UMBRACO__CMS__UNATTENDED__UNATTENDEDUSEREMAIL)
UMBRACO_USER_PASSWORD=$(UMBRACO__CMS__UNATTENDED__UNATTENDEDUSERPASSWORD)
2024-04-22 08:58:05 +02:00
URL=$(ASPNETCORE_URLS)
STORAGE_STAGE_PATH=$(Build.SourcesDirectory)/tests/Umbraco.Tests.AcceptanceTest/playwright/.auth/user.json" | Out-File .env
2024-02-02 10:17:40 +01:00
displayName : Generate .env
workingDirectory : $(Build.SourcesDirectory)/tests/Umbraco.Tests.AcceptanceTest
# Cache and restore NPM packages
2024-01-12 19:32:25 +07:00
- task : Cache@2
2024-02-02 10:17:40 +01:00
displayName : Cache NPM packages
2024-01-12 19:32:25 +07:00
inputs :
2024-02-02 10:17:40 +01:00
key : 'npm_e2e | "$(Agent.OS)" | $(Build.SourcesDirectory)/tests/Umbraco.Tests.AcceptanceTest/package-lock.json'
2024-01-12 19:32:25 +07:00
restoreKeys : |
2024-02-02 10:17:40 +01:00
npm_e2e | "$(Agent.OS)"
npm_e2e
2024-01-12 19:32:25 +07:00
path : $(npm_config_cache)
2024-02-02 10:17:40 +01:00
2024-01-12 19:32:25 +07:00
- script : npm ci --no-fund --no-audit --prefer-offline
2024-02-02 10:17:40 +01:00
workingDirectory : $(Build.SourcesDirectory)/tests/Umbraco.Tests.AcceptanceTest
displayName : Restore NPM packages
# Build application
- pwsh : |
$cmsVersion = "$(Build.BuildNumber)" -replace "\+",".g"
dotnet new nugetconfig
dotnet nuget add source ./nupkg --name Local
dotnet new install Umbraco.Templates::$cmsVersion
dotnet new umbraco --name UmbracoProject --version $cmsVersion --exclude-gitignore --no-restore --no-update-check
dotnet restore UmbracoProject
cp $(Build.SourcesDirectory)/tests/Umbraco.Tests.AcceptanceTest.UmbracoProject/*.cs UmbracoProject
dotnet build UmbracoProject --configuration $(buildConfiguration) --no-restore
dotnet dev-certs https
displayName : Build application
workingDirectory : $(Agent.BuildDirectory)/app
# Run application
- bash : |
nohup dotnet run --project UmbracoProject --configuration $(buildConfiguration) --no-build --no-launch-profile > $(Build.ArtifactStagingDirectory)/playwright.log 2>&1 &
echo "##vso[task.setvariable variable=AcceptanceTestProcessId]$!"
displayName : Run application (Linux)
condition : and(succeeded(), eq(variables['Agent.OS'], 'Linux'))
workingDirectory : $(Agent.BuildDirectory)/app
- pwsh : |
$process = Start-Process dotnet "run --project UmbracoProject --configuration $(buildConfiguration) --no-build --no-launch-profile 2>&1" -PassThru -NoNewWindow -RedirectStandardOutput $(Build.ArtifactStagingDirectory)/playwright.log
Write-Host "##vso[task.setvariable variable=AcceptanceTestProcessId]$($process.Id)"
displayName : Run application (Windows)
2024-01-12 19:32:25 +07:00
condition : and(succeeded(), eq(variables['Agent.OS'], 'Windows_NT'))
2024-02-02 10:17:40 +01:00
workingDirectory : $(Agent.BuildDirectory)/app
# Wait for application to start responding to requests
- pwsh : npx wait-on -v --interval 1000 --timeout 120000 $(ASPNETCORE_URLS)
displayName : Wait for application
workingDirectory : tests/Umbraco.Tests.AcceptanceTest
# Install Playwright and dependencies
- pwsh : npx playwright install --with-deps
displayName : Install Playwright
workingDirectory : tests/Umbraco.Tests.AcceptanceTest
# Test
2024-12-02 14:48:15 +01:00
- pwsh : $(testCommand)
2024-02-02 10:17:40 +01:00
displayName : Run Playwright tests
continueOnError : true
workingDirectory : tests/Umbraco.Tests.AcceptanceTest
env :
CI : true
CommitId : $(Build.SourceVersion)
AgentOs : $(Agent.OS)
# Stop application
- bash : kill -15 $(AcceptanceTestProcessId)
displayName : Stop application (Linux)
condition : and(succeeded(), ne(variables.AcceptanceTestProcessId, ''), eq(variables['Agent.OS'], 'Linux'))
- pwsh : Stop-Process -Id $(AcceptanceTestProcessId)
displayName : Stop application (Windows)
condition : and(succeeded(), ne(variables.AcceptanceTestProcessId, ''), eq(variables['Agent.OS'], 'Windows_NT'))
# Copy artifacts
- pwsh : |
if (Test-Path tests/Umbraco.Tests.AcceptanceTest/results/*) {
Copy-Item tests/Umbraco.Tests.AcceptanceTest/results $(Build.ArtifactStagingDirectory) -Recurse
}
displayName : Copy Playwright results
condition : succeededOrFailed()
# Publish
- task : PublishPipelineArtifact@1
displayName : Publish test artifacts
condition : succeededOrFailed()
inputs :
targetPath : $(Build.ArtifactStagingDirectory)
2024-11-09 10:28:23 +01:00
artifact : "Acceptance Tests - $(Agent.JobName) - Attempt #$(System.JobAttempt)"
2024-02-02 10:17:40 +01:00
- job :
displayName : E2E Tests (SQL Server)
variables :
# Connection string
CONNECTIONSTRINGS__UMBRACODBDSN : Data Source=(localdb)\MSSQLLocalDB;AttachDbFilename=|DataDirectory|\Umbraco.mdf;Integrated Security=True
CONNECTIONSTRINGS__UMBRACODBDSN_PROVIDERNAME : Microsoft.Data.SqlClient
strategy :
matrix :
2024-12-02 14:48:15 +01:00
${{ if eq(parameters.sqlServerLinuxAcceptanceTests, True) }}:
LinuxPart1Of3 :
testCommand : "npx playwright test DefaultConfig --grep \"@smoke\" --grep-invert \"Users\" --shard=1/3"
2024-11-20 12:19:46 +01:00
vmImage : "ubuntu-latest"
SA_PASSWORD : $(UMBRACO__CMS__UNATTENDED__UNATTENDEDUSERPASSWORD)
CONNECTIONSTRINGS__UMBRACODBDSN : "Server=(local);Database=Umbraco;User Id=sa;Password=$(SA_PASSWORD);TrustServerCertificate=True"
2024-12-02 14:48:15 +01:00
LinuxPart2Of3 :
testCommand : "npx playwright test DefaultConfig --grep \"@smoke\" --grep-invert \"Users\" --shard=2/3"
vmImage : "ubuntu-latest"
SA_PASSWORD : $(UMBRACO__CMS__UNATTENDED__UNATTENDEDUSERPASSWORD)
CONNECTIONSTRINGS__UMBRACODBDSN : "Server=(local);Database=Umbraco;User Id=sa;Password=$(SA_PASSWORD);TrustServerCertificate=True"
LinuxPart3Of3 :
testCommand : "npx playwright test DefaultConfig --grep \"@smoke\" --grep-invert \"Users\" --shard=3/3"
vmImage : "ubuntu-latest"
SA_PASSWORD : $(UMBRACO__CMS__UNATTENDED__UNATTENDEDUSERPASSWORD)
CONNECTIONSTRINGS__UMBRACODBDSN : "Server=(local);Database=Umbraco;User Id=sa;Password=$(SA_PASSWORD);TrustServerCertificate=True"
WindowsPart1Of3 :
vmImage : "windows-latest"
testCommand : "npx playwright test DefaultConfig --grep \"@smoke\" --grep-invert \"Users\" --shard=1/3"
WindowsPart2Of3 :
vmImage : "windows-latest"
testCommand : "npx playwright test DefaultConfig --grep \"@smoke\" --grep-invert \"Users\" --shard=2/3"
WindowsPart3Of3 :
2024-11-09 10:28:23 +01:00
vmImage : "windows-latest"
2024-12-02 14:48:15 +01:00
testCommand : "npx playwright test DefaultConfig --grep \"@smoke\" --grep-invert \"Users\" --shard=3/3"
2024-02-02 10:17:40 +01:00
pool :
vmImage : $(vmImage)
steps :
# Setup test environment
- task : DownloadPipelineArtifact@2
displayName : Download NuGet artifacts
inputs :
artifact : nupkg
path : $(Agent.BuildDirectory)/app/nupkg
- task : NodeTool@0
displayName : Use Node.js $(nodeVersion)
inputs :
versionSpec : $(nodeVersion)
2024-01-12 19:32:25 +07:00
- task : UseDotNet@2
2024-02-02 10:17:40 +01:00
displayName : Use .NET SDK from global.json
2024-01-12 19:32:25 +07:00
inputs :
2024-02-02 10:17:40 +01:00
useGlobalJson : true
2024-01-12 19:32:25 +07:00
- pwsh : |
2024-02-02 10:17:40 +01:00
"UMBRACO_USER_LOGIN=$(UMBRACO__CMS__UNATTENDED__UNATTENDEDUSEREMAIL)
UMBRACO_USER_PASSWORD=$(UMBRACO__CMS__UNATTENDED__UNATTENDEDUSERPASSWORD)
2024-09-25 07:04:12 +02:00
URL=$(ASPNETCORE_URLS)
STORAGE_STAGE_PATH=$(Build.SourcesDirectory)/tests/Umbraco.Tests.AcceptanceTest/playwright/.auth/user.json" | Out-File .env
2024-02-02 10:17:40 +01:00
displayName : Generate .env
workingDirectory : $(Build.SourcesDirectory)/tests/Umbraco.Tests.AcceptanceTest
# Cache and restore NPM packages
- task : Cache@2
displayName : Cache NPM packages
inputs :
key : 'npm_e2e | "$(Agent.OS)" | $(Build.SourcesDirectory)/tests/Umbraco.Tests.AcceptanceTest/package-lock.json'
restoreKeys : |
npm_e2e | "$(Agent.OS)"
npm_e2e
path : $(npm_config_cache)
- script : npm ci --no-fund --no-audit --prefer-offline
workingDirectory : $(Build.SourcesDirectory)/tests/Umbraco.Tests.AcceptanceTest
displayName : Restore NPM packages
# Build application
2024-01-12 19:32:25 +07:00
- pwsh : |
2024-02-02 10:17:40 +01:00
$cmsVersion = "$(Build.BuildNumber)" -replace "\+",".g"
dotnet new nugetconfig
dotnet nuget add source ./nupkg --name Local
dotnet new install Umbraco.Templates::$cmsVersion
dotnet new umbraco --name UmbracoProject --version $cmsVersion --exclude-gitignore --no-restore --no-update-check
dotnet restore UmbracoProject
cp $(Build.SourcesDirectory)/tests/Umbraco.Tests.AcceptanceTest.UmbracoProject/*.cs UmbracoProject
dotnet build UmbracoProject --configuration $(buildConfiguration) --no-restore
2024-01-12 19:32:25 +07:00
dotnet dev-certs https
2024-02-02 10:17:40 +01:00
displayName : Build application
workingDirectory : $(Agent.BuildDirectory)/app
# Start SQL Server
- powershell : docker run --name mssql -d -p 1433:1433 -e "ACCEPT_EULA=Y" -e "MSSQL_SA_PASSWORD=$(SA_PASSWORD)" mcr.microsoft.com/mssql/server:2022-latest
displayName : Start SQL Server Docker image (Linux)
condition : and(succeeded(), eq(variables['Agent.OS'], 'Linux'))
- pwsh : SqlLocalDB start MSSQLLocalDB
displayName : Start SQL Server LocalDB (Windows)
condition : and(succeeded(), eq(variables['Agent.OS'], 'Windows_NT'))
# Run application
- bash : |
nohup dotnet run --project UmbracoProject --configuration $(buildConfiguration) --no-build --no-launch-profile > $(Build.ArtifactStagingDirectory)/playwright.log 2>&1 &
echo "##vso[task.setvariable variable=AcceptanceTestProcessId]$!"
displayName : Run application (Linux)
condition : and(succeeded(), eq(variables['Agent.OS'], 'Linux'))
workingDirectory : $(Agent.BuildDirectory)/app
- pwsh : |
$process = Start-Process dotnet "run --project UmbracoProject --configuration $(buildConfiguration) --no-build --no-launch-profile 2>&1" -PassThru -NoNewWindow -RedirectStandardOutput $(Build.ArtifactStagingDirectory)/playwright.log
2024-01-12 19:32:25 +07:00
Write-Host "##vso[task.setvariable variable=AcceptanceTestProcessId]$($process.Id)"
2024-02-02 10:17:40 +01:00
displayName : Run application (Windows)
2024-01-12 19:32:25 +07:00
condition : and(succeeded(), eq(variables['Agent.OS'], 'Windows_NT'))
2024-02-02 10:17:40 +01:00
workingDirectory : $(Agent.BuildDirectory)/app
# Wait for application to start responding to requests
2024-01-12 19:32:25 +07:00
- pwsh : npx wait-on -v --interval 1000 --timeout 120000 $(ASPNETCORE_URLS)
2024-02-02 10:17:40 +01:00
displayName : Wait for application
2024-01-12 19:32:25 +07:00
workingDirectory : tests/Umbraco.Tests.AcceptanceTest
2024-02-02 10:17:40 +01:00
# Install Playwright and dependencies
2024-01-12 19:32:25 +07:00
- pwsh : npx playwright install --with-deps
displayName : Install Playwright
workingDirectory : tests/Umbraco.Tests.AcceptanceTest
2024-02-02 10:17:40 +01:00
# Test
2024-12-02 14:48:15 +01:00
- pwsh : $(testCommand)
2024-02-02 10:17:40 +01:00
displayName : Run Playwright tests
2024-01-12 19:32:25 +07:00
continueOnError : true
workingDirectory : tests/Umbraco.Tests.AcceptanceTest
env :
CI : true
CommitId : $(Build.SourceVersion)
AgentOs : $(Agent.OS)
2024-02-02 10:17:40 +01:00
# Stop application
- bash : kill -15 $(AcceptanceTestProcessId)
displayName : Stop application (Linux)
condition : and(succeeded(), ne(variables.AcceptanceTestProcessId, ''), eq(variables['Agent.OS'], 'Linux'))
- pwsh : Stop-Process -Id $(AcceptanceTestProcessId)
displayName : Stop application (Windows)
condition : and(succeeded(), ne(variables.AcceptanceTestProcessId, ''), eq(variables['Agent.OS'], 'Windows_NT'))
# Stop SQL Server
- pwsh : docker stop mssql
displayName : Stop SQL Server Docker image (Linux)
condition : and(succeeded(), eq(variables['Agent.OS'], 'Linux'))
- pwsh : SqlLocalDB stop MSSQLLocalDB
displayName : Stop SQL Server LocalDB (Windows)
condition : and(succeeded(), eq(variables['Agent.OS'], 'Windows_NT'))
# Copy artifacts
2024-01-12 19:32:25 +07:00
- pwsh : |
2024-02-02 10:17:40 +01:00
if (Test-Path tests/Umbraco.Tests.AcceptanceTest/results/*) {
Copy-Item tests/Umbraco.Tests.AcceptanceTest/results $(Build.ArtifactStagingDirectory) -Recurse
}
displayName : Copy Playwright results
condition : succeededOrFailed()
# Publish
2024-01-12 19:32:25 +07:00
- task : PublishPipelineArtifact@1
displayName : Publish test artifacts
2024-02-02 10:17:40 +01:00
condition : succeededOrFailed()
2024-01-12 19:32:25 +07:00
inputs :
targetPath : $(Build.ArtifactStagingDirectory)
2024-11-09 10:28:23 +01:00
artifact : "Acceptance Tests - $(Agent.JobName) - Attempt #$(System.JobAttempt)"
2022-10-13 15:29:38 +02:00
2022-05-19 10:25:44 +01:00
###############################################
## Release
###############################################
- stage : Deploy_MyGet
displayName : MyGet pre-release
dependsOn :
- Unit
- Integration
2024-09-25 07:04:12 +02:00
# - E2E
2023-10-18 14:52:08 +02:00
condition : and(succeeded(), or(eq(dependencies.Build.outputs['A.build.NBGV_PublicRelease'], 'True'), ${{parameters.myGetDeploy}}))
2022-05-19 10:25:44 +01:00
jobs :
- job :
displayName : Push to pre-release feed
steps :
- checkout : none
- task : DownloadPipelineArtifact@2
displayName : Download nupkg
inputs :
artifact : nupkg
path : $(Build.ArtifactStagingDirectory)/nupkg
2022-10-19 12:51:26 +02:00
- task : NuGetCommand@2
displayName : NuGet push
inputs :
2024-11-09 10:28:23 +01:00
command : "push"
2022-10-19 12:51:26 +02:00
packagesToPush : $(Build.ArtifactStagingDirectory)/**/*.nupkg
2024-11-09 10:28:23 +01:00
nuGetFeedType : "external"
2023-12-05 10:54:23 +01:00
${{ if eq(parameters.isNightly, true) }}:
2024-11-09 10:28:23 +01:00
publishFeedCredentials : "MyGet - Umbraco Nightly"
2023-12-05 10:54:23 +01:00
${{ else }}:
2024-11-09 10:28:23 +01:00
publishFeedCredentials : "MyGet - Pre-releases"
2023-12-05 10:54:23 +01:00
- job :
displayName : Push to pre-release feed (npm)
steps :
- checkout : none
- download : current
artifact : npm
- bash : |
# Check if we are on a nightly build
if [ $isNightly = "False" ]; then
echo "##[debug]Prerelease build detected"
registry="https://www.myget.org/F/umbracoprereleases/npm/"
else
echo "##[debug]Nightly build detected"
registry="https://www.myget.org/F/umbraconightly/npm/"
fi
echo "@umbraco-cms:registry=$registry" >> .npmrc
env :
isNightly : ${{parameters.isNightly}}
workingDirectory : $(Pipeline.Workspace)/npm
displayName : Add scoped registry to .npmrc
- task : npmAuthenticate@0
displayName : Authenticate with npm (MyGet)
inputs :
2024-11-09 10:28:23 +01:00
workingFile : "$(Pipeline.Workspace)/npm/.npmrc"
customEndpoint : "MyGet (npm) - Umbracoprereleases, MyGet (npm) - Umbraconightly"
2023-12-05 10:54:23 +01:00
- bash : |
# Setup temp npm project to load in defaults from the local .npmrc
npm init -y
# Find the first .tgz file in the current directory and publish it
files=( ./*.tgz )
2024-04-18 10:22:11 +02:00
npm publish "${files[0]}"
2023-12-05 10:54:23 +01:00
displayName : Push to npm (MyGet)
workingDirectory : $(Pipeline.Workspace)/npm
2022-05-19 10:25:44 +01:00
- stage : Deploy_NuGet
displayName : NuGet release
dependsOn :
- Deploy_MyGet
2022-08-03 10:45:11 +02:00
- Build_Docs
2023-10-18 14:52:08 +02:00
condition : and(succeeded(), or(eq(dependencies.Build.outputs['A.build.NBGV_PublicRelease'], 'True'), ${{parameters.nuGetDeploy}}))
2022-05-19 10:25:44 +01:00
jobs :
- job :
displayName : Push to NuGet
steps :
- checkout : none
- task : DownloadPipelineArtifact@2
displayName : Download nupkg
inputs :
artifact : nupkg
path : $(Build.ArtifactStagingDirectory)/nupkg
2022-10-05 12:14:43 +02:00
- task : NuGetCommand@2
displayName : NuGet push
2022-05-19 10:25:44 +01:00
inputs :
2024-11-09 10:28:23 +01:00
command : "push"
2022-10-19 14:32:05 +02:00
packagesToPush : $(Build.ArtifactStagingDirectory)/**/*.nupkg
2024-11-09 10:28:23 +01:00
nuGetFeedType : "external"
publishFeedCredentials : "NuGet - Umbraco.*"
2021-03-18 14:48:22 +01:00
2023-12-05 10:54:23 +01:00
- stage : Deploy_Npm
displayName : Npm release
condition : and(succeeded(), or(eq(dependencies.Build.outputs['A.build.NBGV_PublicRelease'], 'True'), ${{parameters.npmDeploy}}))
dependsOn :
- Deploy_NuGet
jobs :
- job : Publish
displayName : Push to NPM
steps :
- checkout : none
- download : current
artifact : npm
- bash : echo "@umbraco-cms:registry=https://registry.npmjs.org" >> .npmrc
workingDirectory : $(Pipeline.Workspace)/npm
displayName : Add scoped registry to .npmrc
- task : npmAuthenticate@0
displayName : Authenticate with npm
inputs :
workingFile : $(Pipeline.Workspace)/npm/.npmrc
2024-11-09 10:28:23 +01:00
customEndpoint : "NPM - Umbraco Backoffice"
2023-12-05 10:54:23 +01:00
- script : |
# Setup temp npm project to load in defaults from the local .npmrc
npm init -y
# Find the first .tgz file in the current directory and publish it
files=( ./*.tgz )
2024-04-18 10:22:11 +02:00
npm publish "${files[0]}"
2023-12-05 10:54:23 +01:00
displayName : Push to npm
workingDirectory : $(Pipeline.Workspace)/npm
2022-05-19 10:25:44 +01:00
- stage : Upload_API_Docs
pool :
2024-11-09 10:28:23 +01:00
vmImage : "windows-latest" # Apparently AzureFileCopy is windows only :(
2022-05-19 10:25:44 +01:00
variables :
2024-01-23 16:38:16 +01:00
umbracoMajorVersion : $[ stageDependencies.Build.A.outputs['build.NBGV_VersionMajor'] ]
2023-08-09 10:20:51 +02:00
displayName : Upload API Documentation
2022-05-19 10:25:44 +01:00
dependsOn :
2024-01-23 16:38:16 +01:00
- Build
2024-04-16 14:56:40 +02:00
- Deploy_NuGet
2023-10-18 14:52:08 +02:00
condition : and(succeeded(), or(eq(dependencies.Build.outputs['A.build.NBGV_PublicRelease'], 'True'), ${{parameters.uploadApiDocs}}))
2022-05-19 10:25:44 +01:00
jobs :
2023-12-05 12:38:14 +01:00
- job :
displayName : Upload C# Docs
steps :
- checkout : none
- task : DownloadPipelineArtifact@2
displayName : Download artifact
inputs :
artifact : csharp-docs
path : $(Build.SourcesDirectory)
- task : ExtractFiles@1
inputs :
archiveFilePatterns : $(Build.SourcesDirectory)/csharp-docs.zip
destinationFolder : $(Build.ArtifactStagingDirectory)/csharp-docs
- task : AzureFileCopy@4
2024-11-09 10:28:23 +01:00
displayName : "Copy C# Docs to blob storage"
2023-12-05 12:38:14 +01:00
inputs :
2024-11-09 10:28:23 +01:00
SourcePath : "$(Build.ArtifactStagingDirectory)/csharp-docs/*"
2023-12-05 12:38:14 +01:00
azureSubscription : umbraco-storage
Destination : AzureBlob
storage : umbracoapidocs
2024-11-09 10:28:23 +01:00
ContainerName : "$web"
2023-12-05 12:38:14 +01:00
BlobPrefix : v$(umbracoMajorVersion)/csharp
2024-01-23 16:43:27 +01:00
CleanTargetBeforeCopy : true
2022-05-19 10:25:44 +01:00
- job :
2024-07-02 09:40:53 +02:00
displayName : Upload Storybook
2022-05-19 10:25:44 +01:00
steps :
- checkout : none
- task : DownloadPipelineArtifact@2
displayName : Download artifact
inputs :
artifact : ui-docs
path : $(Build.SourcesDirectory)
- task : ExtractFiles@1
inputs :
archiveFilePatterns : $(Build.SourcesDirectory)/ui-docs.zip
destinationFolder : $(Build.ArtifactStagingDirectory)/ui-docs
- task : AzureFileCopy@4
2024-11-09 10:28:23 +01:00
displayName : "Copy Storybook to blob storage"
2022-05-19 10:25:44 +01:00
inputs :
2024-11-09 10:28:23 +01:00
SourcePath : "$(Build.ArtifactStagingDirectory)/ui-docs/*"
2022-05-19 10:25:44 +01:00
azureSubscription : umbraco-storage
Destination : AzureBlob
storage : umbracoapidocs
2024-11-09 10:28:23 +01:00
ContainerName : "$web"
2022-05-19 10:25:44 +01:00
BlobPrefix : v$(umbracoMajorVersion)/ui
2024-01-23 16:43:27 +01:00
CleanTargetBeforeCopy : true
2024-07-02 09:40:53 +02:00
- job :
displayName : Upload UI API Docs
steps :
- checkout : none
- task : DownloadPipelineArtifact@2
displayName : Download artifact
inputs :
artifact : ui-api-docs
path : $(Build.SourcesDirectory)
- task : ExtractFiles@1
inputs :
archiveFilePatterns : $(Build.SourcesDirectory)/ui-api-docs.zip
destinationFolder : $(Build.ArtifactStagingDirectory)/ui-api-docs
- task : AzureFileCopy@4
2024-11-09 10:28:23 +01:00
displayName : "Copy UI API Docs to blob storage"
2024-07-02 09:40:53 +02:00
inputs :
2024-11-09 10:28:23 +01:00
SourcePath : "$(Build.ArtifactStagingDirectory)/ui-api-docs/*"
2024-07-02 09:40:53 +02:00
azureSubscription : umbraco-storage
Destination : AzureBlob
storage : umbracoapidocs
2024-11-09 10:28:23 +01:00
ContainerName : "$web"
2024-07-02 09:40:53 +02:00
BlobPrefix : v$(umbracoMajorVersion)/ui-api
CleanTargetBeforeCopy : true
