Commit Graph

24 Commits

Author SHA1 Message Date
Nhu Dinh
7bcef5fde5 Nightly E2E test pipeline: QA Run E2E tests on v17/dev and enable the different setting app tests by default (#20379)
Update yaml file to run the nightly E2E tests against the v17/dev branch and run different app settings tests by default
2025-10-07 15:36:54 +07:00
Nhu Dinh
84fb1f4d6c Extension Registry: QA Added acceptance tests for readOnlyGuards rule, entity action and custom property editor (#19939)
* Added tests for readOnlyGuard rules

* Added backoffice override files for readOnlyGuard tests

* Bumped version

* Added project for ExtensionRegistry tests in playwright configs

* Updated nightly E2E test pipelines to run Extension Registry tests

* Updated nightly E2E test pipeline

* Updated nightly E2E test pipeline

* Updated playwright configs

* Updated nightly E2E test pipeline

* Add test for Entity Action Extension to retrieve entityType and unique (#20020)

* Add entity action test to get unique and entity type

* update test entity action

---------

Co-authored-by: Lan Nguyen Thuy <lnt@umbraco.dk>

* Added job to run the Extension Registry tests in the nightly pipeline

* Cleaned up

* Restructure AdditionSetup folder for extension registry

* Updated yaml file for nightly E2E pipeline

* Updated json file for lock action

* Skip test for content delivery API

* Updated port

* Comment out others to run only extension registry tests

* Updated port

* Remove retrieve action folder to test

* Reverted nightly E2E test pipeline

* Reverted

* Updated umbraco package json

* Reverted

* Renamed AdditionalSetup folder

* Renamed folder

* Added appsetting.json file

* Updated appsettings.json

* Updated appsettings.json

* Added debug step

* Added step to build backoffice

* Reverted

* Only spec.ts file run in the extension registry project

* Property Editor: Add tests for create and using custom property editor (#20213)

* Property Editor: tests for create and using custom property editor

* Update tests/Umbraco.Tests.AcceptanceTest/tests/ExtensionRegistry/PropertyEditorTest.spec.ts

Co-authored-by: Nhu Dinh <150406148+nhudinh0309@users.noreply.github.com>

* Update tests/Umbraco.Tests.AcceptanceTest/tests/ExtensionRegistry/PropertyEditorTest.spec.ts

Co-authored-by: Nhu Dinh <150406148+nhudinh0309@users.noreply.github.com>

* Update tests/Umbraco.Tests.AcceptanceTest/tests/ExtensionRegistry/PropertyEditorTest.spec.ts

Co-authored-by: Nhu Dinh <150406148+nhudinh0309@users.noreply.github.com>

* update review from Nhu

* Update tests/Umbraco.Tests.AcceptanceTest/tests/ExtensionRegistry/CustomPropertyEditor.spec.ts

Co-authored-by: Nhu Dinh <150406148+nhudinh0309@users.noreply.github.com>

* Update tests/Umbraco.Tests.AcceptanceTest/tests/ExtensionRegistry/CustomPropertyEditor.spec.ts

Co-authored-by: Nhu Dinh <150406148+nhudinh0309@users.noreply.github.com>

* fix comment from Nhu

---------

Co-authored-by: Lan Nguyen Thuy <lnt@umbraco.dk>
Co-authored-by: Nhu Dinh <150406148+nhudinh0309@users.noreply.github.com>

* Format code

* Fixed

* Format code

* Format code

* Format code

* Updated indentation

* Fixed comments

* change the name of test

---------

Co-authored-by: NguyenThuyLan <116753400+NguyenThuyLan@users.noreply.github.com>
Co-authored-by: Lan Nguyen Thuy <lnt@umbraco.dk>
Co-authored-by: Andreas Zerbst <andr317c@live.dk>
2025-10-06 09:47:06 +00:00
Andreas Zerbst
f23050d5c6 V16 QA cherry picked acceptance tests with different configuration (#20106)
* Added appsettings

* Added test setup for different config

* Added appsettings for external login

* Added acceptance tests

* Updated pipelines

* Updated solution file

* V15 QA Added external login provider tests and split pipeline into templates (#20049)

* Added setup for external login

* Started on yaml

* Added test file

* Updated pipeline

* Use env vars

* Added env variables and commented out test we don't need to run

* Removed list from matrix

* Updated condition

* Updated package path

* Updated testFolder

* double slash

* Updated condition

* Updated condition again

* Added port

* Removed redundant values

* Set as env vars

* Added env vars beneath matrix

* Get env

* Updated naming

* Updated usage of values

* Added a check for client id, to see if value set

* Moved env out of pool

* Tried moving env

* Trying to fix the env being empty

* Removed env

* Updated name of variable

* Fixed A cyclical reference

* Updated typo

* More logging

* Reverted change

* Added env

* Added env to tests

* Cleaned up

* Added yaml template files

* Updated nighly pipepline to use templates

* Updated sln

* Split yaml into templates for e2e setup

* Updated pipeline

* Updated solution file

* Set value

* Added if statement

* Added variables

* Set default values

* Updated values

* Updated condition

* Run multiple tests

* Added env

* Updated from parameter to variable

* Fixed condition

* Fixed condition to use actual value

* Updated npx wait on command

* Updated pwsh

* Updated port again

* Updated port value

* Updated wait on

* Updated condition

* Restructured

* Updated var

* Updated run application steps

* Added echo

* Updated to boolean

* Updated conditions

* Updated test template usage

* Added databaseType

* Added another databaseType

* Split up templates

* Fixed indentation

* Updated condition

* updated path

* removed build from path

* Updated conditions for azureAd

* Fixed indentation

* Updated to single qoutes

* Cleaned up

* Removed unused file

* Clarified namin

* Moved

* Updated pipeline, not done

* Updated locator

* Updated pipelines

* Updated test helpers package

* Skipped build stage for default app settings tests

* Updated password var

* Updated locators

* Updated defaultconfig build setup

* Split E2E stage in two

* Added parameter for skipping integration tests

* Cleaned up

* Added ASPNETCORE_URLS

* V15 QA acceptance tests with appsettings (#19550)

* Start of appsetting

* Updated setup of playwright

* Adjusted the pipeline

* Updated appsetting

* Added install test

* Added comments

* Updated pipeline

* Updated development app settings

* Commented tests out

* comment

* Added if statement

* Updated pipeline

* Fixed condition

* Changed to production

* Added a log

* Updated copy item

* Added

* Updated app settings

* Updated pipeline

* Moved playwright login

* Updated pipeline

* Updated app setting

* Updated nightly

* Updated appsettings

* Updated get

* Updated wait on

* Updated appsettings

* Updated connection string

* Updates

* Skips code

* Updated variable

* Updated pipeline

* We want to always retain the trace, to see if the test runs as expected on the pipeline

* Added a temporary wait till port is open

* Fixed condition

* Added missing tcp for wait on

* Updated URL env

* Updated setup

* Fixed string

* Updated locator

* Split tests into SQLite and SQLServer

* Updated pipeline to run all tests

* Retain trace on failure

* Added testFolder var

* Added appsettings and program for delivery api tests

* Updated playwright config

* Split test runners into defaultconfig and different app settings

* Added delivery api tests

* Cleaned up tests

* Bumped version

* Updated pipeline

* Small fixes

* Added password

* Updated connection string

* Fixed

* Removed quotes

* Removed unnecessary connection string

* Added missing password

* Cleaned up

* Cleaned up

* Cleaned up

* Updated to use helpers

* Bumped version

* Updated helper usage

* Added password to variables and a condition

* Added check

* Indented value

* Fixed condition

* More updates

* Updated variable

* Removed settings

* Updated delivery api tests

* Bumped version

* Updated test

* Removed unnecessary variables

* Updates based on copilot comments

* Fixed merge conflict

* Fixed env creation step

* Bumped version

* Updated tests to use new helper

* Updated helper

* Updated to string

* Moved logic to conditions

* bumped version

* Use new name for helper

* Remove echo

* Added variable
2025-09-18 10:58:54 +02:00
Andreas Zerbst
aad253fb11 V16 QA add slack webhook for e2e tests (#19866)
* Added stage to notify slackbot

* Update nightly-E2E-test-pipelines.yml for Azure Pipelines

* Fixed indentation

* Update nightly-E2E-test-pipelines.yml for Azure Pipelines

* Updated stage name

* Added issues to condition

* Fixed SucceededWithIssues condition

* Added dependencies to condition

* Text formatting cleanup

* Clarified webhook variable name

* Updated url

* Added another url

* Added URL encoding

* Added missing $

* Added comments
2025-08-06 10:43:35 +00:00
Nhu Dinh
615fcddcf7 V16 QA Updated acceptance tests with skip() and fixme() tag (#19397)
* Updated nightly E2E pipeline

* Fixed failing E2E tests

* Skipped content tests wirh list view content due to an issue

* Updated tests due to UI changes

* Updated fixme and skip tests - part 1

* Removed this file because the tests are already covered elsewhere

* Updated fixme() tests

* Updated skip() tests

* Bumped version

* Bumped version

* Bumped version

* Removed notification verification

* Removed the step to verify the notification for save action

* Fixed the failing tests

* Updated name of permission

* Bumped version

* Fixed failing tests

* Bumped testHelpers

* Removed tests related to tiptap toolbar as they are covered in another class

* Cleaned up

* Added more waits

* Cleaned up

* Added step to ensure redirect URL is created when renaming content.

* Restructured the tests

* Removed unnecessary steps

* Fixed isItemVisibleInRecycleBin

* Fixed isItemVisibleInRecycleBin

* Bumped version

* Added more wait for the deletion to complete

* Added waits for the deletion to complete

* Added more waits

* Removed unnecessary waits

* Added more waits to improve test stability

* Added skip for the flaky test

* Added test for removing a stylesheet in a block grid editor

* Updated test due to api helper changes

* Bumped version

* Fixed failing smoke test

---------

Co-authored-by: Andreas Zerbst <andr317c@live.dk>
2025-08-01 03:30:03 +00:00
Andreas Zerbst
c7dcbfec60 V16 QA Publish Console Error Logs (#19417)
* Updated configuration

* Updated pipeline

* Updated nightly pipeline

* Bumped version

* updated if statement
2025-06-05 11:43:00 +00:00
Andreas Zerbst
3db0ec7efc V16 Updated nightly E2E pipeline to run correct test scripts (#19378)
* Set default to true, so we run the SQlite tests aswell

* Updated test commands to run correct tests

* Disabled Sqlite tests

* Skip flaky test
2025-05-22 11:24:46 +02:00
Andreas Zerbst
df11e438f4 Merge remote-tracking branch 'origin/release/16.0' into v16/merge-16-release-to-main
# Conflicts:
#	build/nightly-E2E-test-pipelines.yml
#	src/Umbraco.Web.UI.Client/eslint.config.js
#	src/Umbraco.Web.UI.Client/package-lock.json
#	src/Umbraco.Web.UI.Client/package.json
#	tests/Umbraco.Tests.AcceptanceTest/package-lock.json
#	tests/Umbraco.Tests.AcceptanceTest/package.json
#	tests/Umbraco.Tests.AcceptanceTest/tests/DefaultConfig/Content/BlockGrid/VariantBlockGrid.spec.ts
#	tests/Umbraco.Tests.AcceptanceTest/tests/DefaultConfig/Content/BlockList/VariantBlockList.spec.ts
#	tests/Umbraco.Tests.AcceptanceTest/tests/DefaultConfig/Content/ContentInfoTab.spec.ts
#	tests/Umbraco.Tests.AcceptanceTest/tests/DefaultConfig/Content/ContentWithCheckboxList.spec.ts
#	tests/Umbraco.Tests.AcceptanceTest/tests/DefaultConfig/Content/ContentWithDropdown.spec.ts
#	tests/Umbraco.Tests.AcceptanceTest/tests/DefaultConfig/Content/ContentWithMediaPicker.spec.ts
#	tests/Umbraco.Tests.AcceptanceTest/tests/DefaultConfig/Content/ContentWithRadiobox.spec.ts
#	tests/Umbraco.Tests.AcceptanceTest/tests/DefaultConfig/Content/ContentWithTextarea.spec.ts
#	tests/Umbraco.Tests.AcceptanceTest/tests/DefaultConfig/Content/ContentWithTextstring.spec.ts
#	tests/Umbraco.Tests.AcceptanceTest/tests/DefaultConfig/Content/RedirectManagement.spec.ts
#	tests/Umbraco.Tests.AcceptanceTest/tests/DefaultConfig/Content/RichTextEditor/ContentWithTiptap.spec.ts
#	tests/Umbraco.Tests.AcceptanceTest/tests/DefaultConfig/Content/RichTextEditor/VariantTipTapBlocks.spec.ts
#	tests/Umbraco.Tests.AcceptanceTest/tests/DefaultConfig/Dictionary/Dictionary.spec.ts
#	version.json
2025-05-20 12:25:14 +02:00
Andreas Zerbst
6a032390c2 Merge remote-tracking branch 'origin/v15/dev' into v16/merge-15-to-16-release
# Conflicts:
#	Directory.Packages.props
#	build/nightly-E2E-test-pipelines.yml
#	src/Umbraco.Core/PropertyEditors/DataValueReferenceFactoryCollection.cs
#	src/Umbraco.Web.UI.Client/package-lock.json
#	src/Umbraco.Web.UI.Client/package.json
#	tests/Umbraco.Tests.AcceptanceTest/package-lock.json
#	tests/Umbraco.Tests.AcceptanceTest/package.json
#	tests/Umbraco.Tests.AcceptanceTest/tests/DefaultConfig/Content/RichTextEditor/ContentWithTiptap.spec.ts
#	tests/Umbraco.Tests.AcceptanceTest/tests/DefaultConfig/Media/Media.spec.ts
#	version.json
2025-05-20 09:26:06 +02:00
Nhu Dinh
ea8ec4e375 V16 QA Updated E2E nightly pipeline (#19355)
Disabled SQLite tests and made it run against the release/16.0 branch
2025-05-20 11:31:14 +07:00
Nhu Dinh
41bf741901 V16 QA Fixing the failing E2E tests (#19318)
* Updated nightly E2E pipeline

* Fixed failing E2E tests

* Skipped content tests wirh list view content due to an issue

* Updated tests due to UI changes

* Bumped version

* Added more waits
2025-05-14 16:10:03 +07:00
Andreas Zerbst
b01def0872 V15 QA added a fix for flaky integration tests run on SQL Server Linux (#18965) 2025-05-08 08:53:29 +02:00
Nhu Dinh
4d7641be97 V16 QA Updated the nightly test pipeline to run against the main branch (#19261) 2025-05-07 10:09:28 +02:00
Nhu Dinh
103631de15 V15 QA Updated the build stage to align with the azure pipelines yml (#19235) 2025-05-06 06:59:32 +00:00
Nhu Dinh
6ce4fbfc0c V16 QA Fixed nightly test pipeline (#19226)
* Use Windows instead

* Updated the build stage to align with the azure pipelines yml

---------

Co-authored-by: Andreas Zerbst <andr317c@live.dk>
2025-05-05 10:13:50 +00:00
Andreas Zerbst
0b0a2dc251 Added v16 to run at midnight (#18785)
Co-authored-by: Nhu Dinh <150406148+nhudinh0309@users.noreply.github.com>
2025-03-25 11:06:33 +07:00
Nhu Dinh
2ebc6badd7 V15 QA Publish E2E test results in the Azure pipeline (#18498)
* Updated playwright config to export the Junit report

* Published the Junit report

* Fixed

* Updated outputDir

* Make one test failed

* Changed npm command

* Fixed outputFile

* Updated folder to export test result

* Make another tests failed

* Updated publish test results job

* Changed testSQLite command

* Updated testRunTitle

* Changed npm command

* Updated testRunTitle

* Reverted

* Fixed comment

* Make some tests failed to test

* Reverted
2025-03-03 13:11:07 +07:00
Andreas Zerbst
b8ee161e08 V15 QA updated our E2E pipeline to output test artifacts in a clearer way (#17826)
* Updated pipeline to publish acceptance test artifacts for each os. and to run tests from npm command

* Adjustment to the targetPath

* Publish all in results folder

* Moved folder

* Reverted changes

* Added all results in test folder

* Updated naming

* Used semicolon

* Additional semicolons

---------

Co-authored-by: Niels Lyngsø <nsl@umbraco.dk>
2025-01-15 10:21:36 +01:00
Andreas Zerbst
303758d1b2 V15 QA add parallelization of e2e tests (#17691)
* Tries parallelizing the E2E tests

* Updated variable

* Updated SqlServer

* Updated playwright config

* Bumped to use 3 shards

* Implemented shards on nightly pipeline

* Fixed indentation

* Added a smokeTest

* Updated test to run

* Added test command

* Update Templates.spec.ts
2024-12-02 14:48:15 +01:00
Andreas Zerbst
126bce7721 V14 QA Skip Users tests on Sqlite (#17330)
* Split sqlite test because we run into db locks

* Uses the new command

(cherry picked from commit 728dc89909)
2024-10-22 13:13:42 +02:00
Andreas Zerbst
e4dacf5c8c V14 QA fixed E2E tests for SQL server (#17122)
* Updated version of test helpers

* Added option to run smoke tests

* Found the issue

* Fixed pipeline

* Removed duplicate file creation

* Removed

* Always run sql server tests

* Removed unused parameter

* Enables sqlServer E2E to run on pipeline

* Removed comment
2024-09-25 07:04:12 +02:00
Andreas Zerbst
e92b4f3172 V15 QA Enabled Nightly E2E Pipeline to run on V15 (#17103)
* Uncommented

* Added timeout

(cherry picked from commit 142db8c0fb)
2024-09-23 14:00:32 +02:00
Andreas Zerbst
d12dedb526 V14 QA update fetch depth (#17068)
* Removed fetch depth

* Added fetch depth 0
2024-09-17 10:16:29 +07:00
Andreas Zerbst
48e4cdaa88 V14 QA new E2E test pipeline (#17064)
* Added E2E pipeline

* Set up CI with Azure Pipelines

[skip ci]

* Update nightly-E2E-test-pipelines.yml for Azure Pipelines

* Updated solution file

* Added fetch depth

* Update nightly-E2E-test-pipelines.yml for Azure Pipelines

* Added a step for installing wait-on

* Added runSqlServerE2ETests

* Update nightly-E2E-test-pipelines.yml for Azure Pipelines

* Update nightly-E2E-test-pipelines.yml for Azure Pipelines

* Updated pipeline so we only run smoke

* Update nightly-E2E-test-pipelines.yml for Azure Pipelines

* Update nightly-E2E-test-pipelines.yml for Azure Pipelines

* Removed pipeline
2024-09-16 10:13:26 +02:00