Commit Graph

276 Commits

Author SHA1 Message Date
Sebastiaan Janssen
cbfeba0807 Fix CodeQL duplicate "permissions" node and reformat 2022-09-20 16:03:57 +02:00
Sebastiaan Janssen
40035aa629 Merge branch 'v10/dev' into v11/dev 2022-09-20 09:13:22 +02:00
Sebastiaan Janssen
85e31bd12d Merge branch 'v10/dev' into v10/contrib
# Conflicts:
#	src/Umbraco.Core/EmbeddedResources/Lang/en.xml
#	src/Umbraco.Core/EmbeddedResources/Lang/en_us.xml
2022-09-19 16:48:36 +02:00
Sebastiaan Janssen
db5d05d641 Revert files that shouldn't change back in v11 2022-09-19 16:37:24 +02:00
Sebastiaan Janssen
87c06f5f98 Back to 10.2.0-ish 2022-09-19 16:14:16 +02:00
Bjarke Berg
74c45ee60d Merge remote-tracking branch 'origin/v10/dev' into v11/dev
# Conflicts:
#	src/Umbraco.Core/DependencyInjection/UmbracoBuilder.Configuration.cs
#	src/Umbraco.Core/Models/Mapping/DictionaryMapDefinition.cs
#	src/Umbraco.Web.BackOffice/Install/InstallApiController.cs
#	version.json
2022-09-12 16:47:20 +02:00
Alex
df3777f976 Update add-issues-to-review-project.yml
Signed-off-by: sashashura <93376818+sashashura@users.noreply.github.com>
2022-09-05 01:45:07 +02:00
Sebastiaan Janssen
17d72f9436 Update 01_bug_report.yml 2022-09-02 09:33:09 +02:00
neilnaveen
fcd1c22bf8 chore: Set permissions for GitHub actions
Restrict the GitHub token permissions only to the required ones; this way, even if the attackers will succeed in compromising your workflow, they won’t be able to do much.

- Included permissions for the action. https://github.com/ossf/scorecard/blob/main/docs/checks.md#token-permissions

https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#permissions

https://docs.github.com/en/actions/using-jobs/assigning-permissions-to-jobs

[Keeping your GitHub Actions and workflows secure Part 1: Preventing pwn requests](https://securitylab.github.com/research/github-actions-preventing-pwn-requests/)

Signed-off-by: neilnaveen <42328488+neilnaveen@users.noreply.github.com>
2022-08-30 18:52:08 +02:00
Mole
748fb7d1f7 Implement new backoffice installer (#12790)
* Add new BackOfficeApi project

* Add swagger

* Add and route new install controller

* Add new install steps

* Add Setup endpoint

* Add missing RequiresExecution methods

* Fix nullability of databasemodel

* Move user information to separate model

* Remove ping method

* Add view models install data

* Move mapping folder

* Move ViewModels

* Add settings endpoint

* Remove unused binderprovider

* Postfix RequiresExecution with async

* Update NewDatabaseUpgradeStep to not depend on install step

* Add installstep collection

* Move registration into backoffice project

* Add InstallService

* Use service in controller

* Add upgrade to install service and use in controller

* Correctly check is database is configured

* Reorganize

* Reorganize into new core and infrastructure

* Rename steps

* Rename BackofficeApi to MangementApi

* Make install step an interface instead of abstract class

* Rename InstallStep to create CreateUserStep

* Move restart runtime and sign in user into install steps

* Move install service into new core project

* Map controllers in composer

* Restrict access to installcontroller based on runtime level

* Use FireAndForget when logging install

* Use actionresult instead of iactionresult

* Set new projects as not packable

* Link to backoffice in 201 response when installed

* Register installations

* Add custom backoffice routing template token

* Move umbraco path trimming out of application convention

* Make it easier to route to backoffice api

* Make swagger version aware and move behind backoffice path

* Obsolete old install classes

* Move maps into single file

This is all mappint to/from viewmodels in some manner

* Remove usage of InstallSetupResult

* Move new projects to the src folder

* Remove InstallationType from IInstallStep

This upgrade steps should implement their own IUpgradeStep interface

* Remove upgrade from service and controller

This should be its own service and controller

* Add xml docs

* Remove internals visible to

* Disable package validation for new projects

Quite the gotcha here, if the projects are brand new, there is no nuget packages to compare with, this causes the build to fail.

* Add ValidateDatabase endpoint

* Remove project references to new backoffice

We don't actually want to depend on this yet, it's just needed for testing/development

* Obsolete installationtype

* Add DatabaseSettingsFactory tests

* Add InstallServiceTests

* Fix InstallServiceTests

* Test RequireRuntimeLevelAttribute

* Implement new backoffice upgrader (#12818)

* Add UpgradeSettingsModel and viewmodel

* Add upgrade/settings endpoint

* Implement upgrade steps

* Add upgrade step collection

* Add UpgradeService

* Add authorize endpoint to UpgradeController

* Fix interface

* Add upgrade service tests

* Remove runtime check in databaseinstallstep

* Move RequireRuntimeLevel to controller

* Add a readme to the new backoffice part

* BackOffice not Backoffice

* Add conditional project references

* Fixes based on review

* Fix up

* Move running of steps into its own method in UpgradeService

* Make services transient

* More fixup

* Log exceptions when running steps
2022-08-29 09:50:48 +02:00
Ronald Barendse
143fbef3d7 Use .NET 7.x in CodeQL build 2022-08-26 09:18:17 +02:00
Jacob Overgaard
173c231859 upgrade setup-dotnet to v2 2022-08-19 14:36:19 +02:00
Jacob Overgaard
c7813aa354 upgrade codeql tasks to v2 2022-08-19 14:35:43 +02:00
Jacob Overgaard
c4c27ec72b add GITHUB_TOKEN restrictive permissions 2022-08-19 14:35:21 +02:00
Jacob Overgaard
65c828390c CodeQL should not scan compiled assets (#12861)
* add the Umbraco.Cms.StaticAssets folder to ignored paths for the codeql scanner

* exclude only the 'wwwroot' folder since other assets could be valid to scan
2022-08-19 11:10:45 +02:00
Sebastiaan Janssen
41eb412403 Create issues-first-response.yml 2022-07-18 09:48:13 +02:00
Sebastiaan Janssen
e626fca243 Merge branch 'v10/dev' into v10/contrib
# Conflicts:
#	src/Umbraco.Core/Actions/ActionAssignDomain.cs
#	src/Umbraco.Core/Actions/ActionCopy.cs
#	src/Umbraco.Core/Actions/ActionCreateBlueprintFromContent.cs
#	src/Umbraco.Core/Actions/ActionDelete.cs
#	src/Umbraco.Core/Actions/ActionMove.cs
#	src/Umbraco.Core/Actions/ActionNew.cs
#	src/Umbraco.Core/Actions/ActionNotify.cs
#	src/Umbraco.Core/Actions/ActionProtect.cs
#	src/Umbraco.Core/Actions/ActionRestore.cs
#	src/Umbraco.Core/Actions/ActionRights.cs
#	src/Umbraco.Core/Actions/ActionRollback.cs
#	src/Umbraco.Core/Actions/ActionSort.cs
#	src/Umbraco.Core/Actions/ActionToPublish.cs
#	src/Umbraco.Core/Actions/ActionUnpublish.cs
#	src/Umbraco.Core/Actions/ActionUpdate.cs
#	src/Umbraco.Core/Constants-Telemetry.cs
#	src/Umbraco.Core/DependencyInjection/ServiceCollectionExtensions.cs
#	src/Umbraco.Core/DependencyInjection/UmbracoBuilder.CollectionBuilders.cs
#	src/Umbraco.Core/DependencyInjection/UmbracoBuilder.Collections.cs
#	src/Umbraco.Core/Models/Trees/ActionMenuItem.cs
#	src/Umbraco.Core/Models/Trees/CreateChildEntity.cs
#	src/Umbraco.Core/Models/Trees/ExportMember.cs
#	src/Umbraco.Core/Models/Trees/MenuItem.cs
#	src/Umbraco.Core/Models/Trees/RefreshNode.cs
#	src/Umbraco.Core/Persistence/Repositories/IRedirectUrlRepository.cs
#	src/Umbraco.Core/PropertyEditors/NestedContentConfiguration.cs
#	src/Umbraco.Core/Routing/ContentFinderByRedirectUrl.cs
#	src/Umbraco.Core/Services/IFileService.cs
#	src/Umbraco.Core/Services/IRedirectUrlService.cs
#	src/Umbraco.Core/Services/RedirectUrlService.cs
#	src/Umbraco.Core/Trees/MenuItemList.cs
#	src/Umbraco.Infrastructure/Persistence/Repositories/Implement/MemberRepository.cs
#	src/Umbraco.Infrastructure/Persistence/Repositories/Implement/RedirectUrlRepository.cs
#	src/Umbraco.Infrastructure/Telemetry/Providers/SystemInformationTelemetryProvider.cs
#	src/Umbraco.Web.BackOffice/Controllers/CodeFileController.cs
#	src/Umbraco.Web.BackOffice/Controllers/ContentController.cs
#	src/Umbraco.Web.BackOffice/Controllers/DictionaryController.cs
#	src/Umbraco.Web.BackOffice/Controllers/EntityController.cs
#	src/Umbraco.Web.BackOffice/Controllers/PreviewController.cs
#	src/Umbraco.Web.BackOffice/Trees/ContentBlueprintTreeController.cs
#	src/Umbraco.Web.BackOffice/Trees/ContentTreeController.cs
#	src/Umbraco.Web.BackOffice/Trees/ContentTreeControllerBase.cs
#	src/Umbraco.Web.BackOffice/Trees/ContentTypeTreeController.cs
#	src/Umbraco.Web.BackOffice/Trees/DataTypeTreeController.cs
#	src/Umbraco.Web.BackOffice/Trees/DictionaryTreeController.cs
#	src/Umbraco.Web.BackOffice/Trees/FileSystemTreeController.cs
#	src/Umbraco.Web.BackOffice/Trees/MacrosTreeController.cs
#	src/Umbraco.Web.BackOffice/Trees/MediaTreeController.cs
#	src/Umbraco.Web.BackOffice/Trees/MediaTypeTreeController.cs
#	src/Umbraco.Web.BackOffice/Trees/MemberGroupTreeController.cs
#	src/Umbraco.Web.BackOffice/Trees/MemberTreeController.cs
#	src/Umbraco.Web.BackOffice/Trees/MemberTypeAndGroupTreeControllerBase.cs
#	src/Umbraco.Web.BackOffice/Trees/RelationTypeTreeController.cs
#	src/Umbraco.Web.BackOffice/Trees/TemplatesTreeController.cs
#	src/Umbraco.Web.UI.Client/src/less/navs.less
#	templates/UmbracoProject/.gitignore
#	tests/Umbraco.Tests.Integration/Umbraco.Core/Telemetry/TelemetryServiceTests.cs
#	tests/Umbraco.Tests.UnitTests/Umbraco.Core/Services/UserDataServiceTests.cs
#	tests/Umbraco.Tests.UnitTests/Umbraco.Core/Telemetry/SystemInformationTelemetryProviderTests.cs
2022-07-11 15:04:46 +02:00
Matt Brailsford
23e94ffdad Add TOC to Contributing page (#12587) 2022-06-17 17:23:43 +10:00
Paul Johnson
f7bf6d5959 Prevent codeql build failures 2022-05-19 10:47:31 +01:00
Sebastiaan Janssen
1422aa196a Add new issues to a project board 2022-05-17 19:30:42 +02:00
Sebastiaan Janssen
230d3e53ad Fix action name 2022-05-17 14:27:00 +02:00
Sebastiaan Janssen
082d853056 Add up-for-grabs workflow - improve log messages in pr-first-response 2022-05-17 14:24:07 +02:00
Sebastiaan Janssen
bbffac22b4 Update BUILD.md 2022-05-17 11:39:58 +02:00
Sebastiaan Janssen
4d41fd77c9 Update BUILD.md 2022-05-17 11:38:02 +02:00
Sebastiaan Janssen
048193e3a5 Update BUILD.md 2022-05-16 13:12:08 +02:00
Sebastiaan Janssen
898c5cc397 Update BUILD.md 2022-05-16 13:08:33 +02:00
failureflawless
2c34751fd4 Updated docs and VSCode build task
as we discovered some changes we needed in the docs
2022-05-13 11:35:52 +02:00
Joe Glombek
953e6f2e34 Rework and update the contributing guide (#12165) 2022-05-12 12:19:44 +02:00
Sebastiaan Janssen
77ebc22b5b Merge branch 'v9/dev' into v10/dev 2022-04-26 10:35:09 +02:00
Sebastiaan Janssen
c1d8052ad6 Merge branch 'v9/dev' into v10/contrib 2022-04-26 10:34:35 +02:00
Sebastiaan Janssen
d5371a15f5 Fix PR first response to work for outside contributors too 2022-04-26 10:28:00 +02:00
Sebastiaan Janssen
6d0a0fff9c Merge branch 'v9/dev' into v10/dev 2022-04-21 14:37:28 +02:00
Sebastiaan Janssen
16a02e0827 Update pr-first-response.yml 2022-04-21 14:36:19 +02:00
Bjarke Berg
23a49161d6 Merge remote-tracking branch 'origin/v9/dev' into v10/dev
# Conflicts:
#	build/build.ps1
#	build/templates/UmbracoPackage/.template.config/template.json
#	build/templates/UmbracoProject/.template.config/dotnetcli.host.json
#	build/templates/UmbracoProject/.template.config/ide.host.json
#	build/templates/UmbracoProject/.template.config/template.json
#	src/Umbraco.Core/Constants-System.cs
#	src/Umbraco.Infrastructure/DependencyInjection/UmbracoBuilder.Services.cs
#	src/Umbraco.Infrastructure/Security/BackOfficeUserStore.cs
#	src/Umbraco.Web.BackOffice/Controllers/AuthenticationController.cs
#	umbraco-netcore-only.sln
2022-04-20 11:09:28 +02:00
Sebastiaan Janssen
184397a327 Adds PR first response action 2022-04-15 19:14:23 +02:00
Sebastiaan Janssen
5bac054311 Add discord badge 2022-04-07 16:36:30 +02:00
Sebastiaan Janssen
4469191c99 Updating some more obsolete Code of Conduct usages 2022-03-18 13:40:03 +01:00
Sebastiaan Janssen
ead813989d Code of conduct has been moved to an organization-wide repository 2022-03-18 13:36:40 +01:00
Paul Johnson
3961c4c233 v10 SQLite support + distributed locking abstractions (#11922)
* Created Persistence.SQLite project skeleton.

* SQLite database initialization

* Various changes and hacks to make things work.

* WIP integration tests

* Fix thread safety tests

* Fix tests that relied on tie breaker sorting.

Spent a fair amount of time looking for a less lazy fix but gave up.

* Convert right join to left join ContentTypeRepository.PerformGetByQuery
SQLite doesn't support right join

* Fix test Can_Generate_Delete_SubQuery_Statement

Worth noting that NPoco.DatabaseTypes.SQLiteDatabaseType doesn't override
EscapeSqlIdentifier so NPoco will escape with [].

SQLite docs say > "A keyword enclosed in square brackets is an identifier.
This is not standard SQL.
This quoting mechanism is used by MS Access and SQL Server and is
included in SQLite for compatibility."

Also could have updated SqliteSyntaxProvider to match npoco but
decided against it.

* Fixes for paginated custom order by

* Fix tests broken by lack of unique indexes.

* Fix SqlServerTableByTableTest tests.

These tests didn't actually do anything as the tables already exist so schema creator just returned.

Did however point out that the default implementation for DoesTableExist just returns false so added a default naive implementation.

* Fix ValidateLoginSession - SelectTop must come later

* dry up database cleanup

* Fix up db migration tests.
We can't drop pk in sqlite without recreating table.
Test looks to be testing that add column works as intended which we can test.

* Prevent schema creation errors.

* SQLite ignore lock tests, WAL back on.

* Fix package schema tests

* Fix NPocoFetchTests - case sensitivity not under test

* Fix AdvancedMigrationTests (where possible)

Migrations probably need a good look later.
Maybe nuke old migrations and only support moving to v10 from v9.

If we do that can do some cleanup.

* Cleanup test database configuration

* Run integration tests against SQLite on build agent.

* Drop MS.Data.SQLite
System.Data.SQLite was quicker to roll out due to more CLR type mapping

* YAML

* Skip Umbraco.Tests.Integration.SqlCe

* Drop SqlServerTableByTable tests.

Until this week they did nothing anyway as they with NewSchemaPerTest
so the tests all passed as CreateTable was no op (already exists).

Also all of the tables are created in an empty database by SchemaValidationTest.cs
DatabaseSchemaCreation_Produces_DatabaseSchemaResult_With_Zero_Errors

* Might aswell run against macOS also.

* Copy azure pipelines task header layout

* Delete SQLCe projects

* Remove SQL CE specific code.

* Remove SQL CE NuSpec, template params, build script setup

* Delete umbraco-netcore-only.sln

* Add SkipTests solution configuration and use for codeql

* Remove reference to deleted nuspec file.

* Refactor ConnectionStrings WRT DataDirectory placeholder & ProviderName.

At this point you can try out SQLite support by setting the following
in appsettings.json and then completing the install process.

"ConnectionStrings": {
  "umbracoDbDSN": "Data Source=|DataDirectory|/umbraco.sqlite",
  "umbracoDbDSN_ProviderName": "System.Data.SQLite"
},

Not currently possible via installer UI without provider name pre-set in
configuration.

* Switch to Microsoft.Data.Sqlite

Some gross hacks but will be good to find out if this works
with apple silicon.

* Enable selection of SQLite via installer UI (also quick install)

* Remove SqlServerDbProviderFactoryCreator to cleanup a TODO

* Move SQL Server support to its own class library

* Add persistence dependencies to Umbraco.CMS metapackage

* Bugfix packages delete query

Created invalid query for SQLite.

* Try out cypress tests Linux + SQLite

* Prevent cypress test artifact upload failure on attempt 2+

* LocalDb bugfixes

* Drop redundant enum

* Move SqlClient constant

* Misc whitespace

* Remove IsSqlCe extension (TODO: drop non 9->10 migrations later).

* Umbraco.Persistence.* -> Umbraco.Cms.Persistence.*

* Display quick install defaults and per provider default database name.

* Misc remove old comment

* little re-arrange

* Remove almost all usages of IsSqlite extension.

* visual adjustments

* Custom Database Configuration is last step and should then say Install.

* use text instead of disabled inputs

* move legend, rename to Install

* Update SqlMainDomLock to work without distributed locks.

* Added IDistributedLockingMechanism interface and in memory impl.

* Drop locking from ISqlSyntaxProvider & wire up scope to abstraction.

* Added SqlServerDistributedLockingMechanism

* Move distributed locking interfaces and exceptions to Core + xmldocs.

* Fix tests, Misc cleanup, Add SQL distributed locking integration tests

* Provide mechanism to specify DistributedLockingMechanism in config
(even if added by composer)

* Nomplementation -> NoImplementation

* Fix misleading comment

* Integration tests use SqlServerDistributedLockingMechanism when possible

* Handle up-gradable locks SqlServerDistributedLockingMechanism.
TODO: InMemoryDistributedLockingMechanism.

Note: Nuked SqlServerDistributedLockingMechanismTests, will still sleep
at night.
Is covered by Umbraco.Cms.Tests.Integration.Umbraco.Infrastructure.Persistence.LockTests

* Make tests pass for InMemoryDistributedLockingMechanism, pretty hacky.

* Tweak constraints on WithCollectionBuilder so i can drop bad constructor

* Added SqliteDistributedLockingMechanism

* Dropped InMemoryDistributedMechanism + magic

InMemoryDistributedMechanism was pretty rubbish and now we have
a decent implementation for SQLite as we no longer block readers
see 8d1f42b.

Also drop the CollectionBuilder setup, instead do the same as we do
for syntax providers etc, it's more automagical so we never require an
explicit selection although we are allowing for it.

However keeping the optional IUmbracoBuilder constructor param for
CollectionBuilders as it's extremely useful.

* Fix quick install "" database name.

* Hide Database Configuration section when a connection string is pre-set.

Doesn't seem worth it to extract db name from connection string.

* Ensure wal test 2+

* Fix logging inconsistencies.

* Ensure in transaction when obtaining locks + no-op the SQLite read lock.

There's no point in running the query just to make a single test pass.

* Fix installer database display names

* Allow SQLite shared cache without losing deferred transactions

* Opt into shared cache for new SQLite databases + fix filename

* Fix misc inconsistency in .gitignore

* Prefer our interceptor interface

* Restore DEBUG_DATABASES code OnConnectionOpened in case it's used.

* Back to private cache.

* Added retry strategy for SQLite + refactor out SQL server specific stuff

* Fix SQL server tests.

* Misc - Orphaned comment, incorrect casing.

* InMemory SQLite test database & turn shared cache back on everywhere.

Co-authored-by: Niels Lyngsø <niels.lyngso@gmail.com>
2022-03-11 17:14:20 +01:00
Bjarke Berg
981e66a673 Merge remote-tracking branch 'origin/v9/dev' into v10/dev
# Conflicts:
#	src/Umbraco.Core/Telemetry/TelemetryService.cs
#	src/Umbraco.Infrastructure/DependencyInjection/UmbracoBuilder.CoreServices.cs
#	src/Umbraco.Infrastructure/DependencyInjection/UmbracoBuilder.Services.cs
#	src/Umbraco.Infrastructure/HostedServices/ScheduledPublishing.cs
#	src/Umbraco.Infrastructure/Migrations/Upgrade/UmbracoPlan.cs
#	src/Umbraco.Infrastructure/Runtime/SqlMainDomLock.cs
#	src/Umbraco.Web.BackOffice/Controllers/ContentController.cs
#	src/Umbraco.Web.BackOffice/Controllers/DashboardController.cs
#	tests/Umbraco.Tests.UnitTests/Umbraco.Infrastructure/HostedServices/ScheduledPublishingTests.cs
2022-03-10 14:36:30 +01:00
Sebastiaan Janssen
411fb0c120 Update CODE_OF_CONDUCT.md 2022-02-23 09:56:21 +01:00
Sebastiaan Janssen
27c60277f2 Update CODE_OF_CONDUCT.md 2022-02-23 09:25:40 +01:00
Bjarke Berg
e92abf7d51 Merge remote-tracking branch 'origin/v9/dev' into v10/dev
# Conflicts:
#	.github/workflows/codeql-analysis.yml
#	build/templates/UmbracoPackage/.template.config/template.json
#	build/templates/UmbracoProject/.template.config/template.json
#	src/Directory.Build.props
#	src/Umbraco.Infrastructure/Security/BackOfficeUserStore.cs
2022-01-23 13:46:10 +01:00
Niels Lyngsø
e204c5bab9 Media Previews (#11888)
Co-authored-by: Niels Lyngsø <nsl@umbraco.com>
Co-authored-by: Mads Rasmussen <madsr@hey.com>
Co-authored-by: Paul Johnson <pmj@umbraco.com>
2022-01-20 16:47:00 +01:00
Paul Johnson
ea3ceab4e2 GH actions - install latest dotnet 2022-01-18 11:51:42 +00:00
Paul Johnson
576f90ad37 Speedup codeql run (#11877)
* Speedup codeql run

* npm build once, ignore node_modules

* Misc cleanup
2022-01-18 00:35:53 +00:00
Paul Johnson
9a145537aa Fix config file, should be on init 2022-01-17 22:29:42 +00:00
Paul Johnson
c410f78986 Restore config file and prevent duplicate run for feature branches 2022-01-17 21:41:24 +00:00
Paul Johnson
2c44d67686 Modify codeql setup. (#11876) 2022-01-17 14:42:41 +00:00
Sebastiaan Janssen
6f214fc269 Update 01_bug_report.yml 2021-11-04 08:47:02 +01:00