yv01p/Umbraco-CMS
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

v10 SQLite support + distributed locking abstractions (#11922)

Browse Source 
* Created Persistence.SQLite project skeleton.

* SQLite database initialization

* Various changes and hacks to make things work.

* WIP integration tests

* Fix thread safety tests

* Fix tests that relied on tie breaker sorting.

Spent a fair amount of time looking for a less lazy fix but gave up.

* Convert right join to left join ContentTypeRepository.PerformGetByQuery
SQLite doesn't support right join

* Fix test Can_Generate_Delete_SubQuery_Statement

Worth noting that NPoco.DatabaseTypes.SQLiteDatabaseType doesn't override
EscapeSqlIdentifier so NPoco will escape with [].

SQLite docs say > "A keyword enclosed in square brackets is an identifier.
This is not standard SQL.
This quoting mechanism is used by MS Access and SQL Server and is
included in SQLite for compatibility."

Also could have updated SqliteSyntaxProvider to match npoco but
decided against it.

* Fixes for paginated custom order by

* Fix tests broken by lack of unique indexes.

* Fix SqlServerTableByTableTest tests.

These tests didn't actually do anything as the tables already exist so schema creator just returned.

Did however point out that the default implementation for DoesTableExist just returns false so added a default naive implementation.

* Fix ValidateLoginSession - SelectTop must come later

* dry up database cleanup

* Fix up db migration tests.
We can't drop pk in sqlite without recreating table.
Test looks to be testing that add column works as intended which we can test.

* Prevent schema creation errors.

* SQLite ignore lock tests, WAL back on.

* Fix package schema tests

* Fix NPocoFetchTests - case sensitivity not under test

* Fix AdvancedMigrationTests (where possible)

Migrations probably need a good look later.
Maybe nuke old migrations and only support moving to v10 from v9.

If we do that can do some cleanup.

* Cleanup test database configuration

* Run integration tests against SQLite on build agent.

* Drop MS.Data.SQLite
System.Data.SQLite was quicker to roll out due to more CLR type mapping

* YAML

* Skip Umbraco.Tests.Integration.SqlCe

* Drop SqlServerTableByTable tests.

Until this week they did nothing anyway as they with NewSchemaPerTest
so the tests all passed as CreateTable was no op (already exists).

Also all of the tables are created in an empty database by SchemaValidationTest.cs
DatabaseSchemaCreation_Produces_DatabaseSchemaResult_With_Zero_Errors

* Might aswell run against macOS also.

* Copy azure pipelines task header layout

* Delete SQLCe projects

* Remove SQL CE specific code.

* Remove SQL CE NuSpec, template params, build script setup

* Delete umbraco-netcore-only.sln

* Add SkipTests solution configuration and use for codeql

* Remove reference to deleted nuspec file.

* Refactor ConnectionStrings WRT DataDirectory placeholder & ProviderName.

At this point you can try out SQLite support by setting the following
in appsettings.json and then completing the install process.

"ConnectionStrings": {
  "umbracoDbDSN": "Data Source=|DataDirectory|/umbraco.sqlite",
  "umbracoDbDSN_ProviderName": "System.Data.SQLite"
},

Not currently possible via installer UI without provider name pre-set in
configuration.

* Switch to Microsoft.Data.Sqlite

Some gross hacks but will be good to find out if this works
with apple silicon.

* Enable selection of SQLite via installer UI (also quick install)

* Remove SqlServerDbProviderFactoryCreator to cleanup a TODO

* Move SQL Server support to its own class library

* Add persistence dependencies to Umbraco.CMS metapackage

* Bugfix packages delete query

Created invalid query for SQLite.

* Try out cypress tests Linux + SQLite

* Prevent cypress test artifact upload failure on attempt 2+

* LocalDb bugfixes

* Drop redundant enum

* Move SqlClient constant

* Misc whitespace

* Remove IsSqlCe extension (TODO: drop non 9->10 migrations later).

* Umbraco.Persistence.* -> Umbraco.Cms.Persistence.*

* Display quick install defaults and per provider default database name.

* Misc remove old comment

* little re-arrange

* Remove almost all usages of IsSqlite extension.

* visual adjustments

* Custom Database Configuration is last step and should then say Install.

* use text instead of disabled inputs

* move legend, rename to Install

* Update SqlMainDomLock to work without distributed locks.

* Added IDistributedLockingMechanism interface and in memory impl.

* Drop locking from ISqlSyntaxProvider & wire up scope to abstraction.

* Added SqlServerDistributedLockingMechanism

* Move distributed locking interfaces and exceptions to Core + xmldocs.

* Fix tests, Misc cleanup, Add SQL distributed locking integration tests

* Provide mechanism to specify DistributedLockingMechanism in config
(even if added by composer)

* Nomplementation -> NoImplementation

* Fix misleading comment

* Integration tests use SqlServerDistributedLockingMechanism when possible

* Handle up-gradable locks SqlServerDistributedLockingMechanism.
TODO: InMemoryDistributedLockingMechanism.

Note: Nuked SqlServerDistributedLockingMechanismTests, will still sleep
at night.
Is covered by Umbraco.Cms.Tests.Integration.Umbraco.Infrastructure.Persistence.LockTests

* Make tests pass for InMemoryDistributedLockingMechanism, pretty hacky.

* Tweak constraints on WithCollectionBuilder so i can drop bad constructor

* Added SqliteDistributedLockingMechanism

* Dropped InMemoryDistributedMechanism + magic

InMemoryDistributedMechanism was pretty rubbish and now we have
a decent implementation for SQLite as we no longer block readers
see 8d1f42b.

Also drop the CollectionBuilder setup, instead do the same as we do
for syntax providers etc, it's more automagical so we never require an
explicit selection although we are allowing for it.

However keeping the optional IUmbracoBuilder constructor param for
CollectionBuilders as it's extremely useful.

* Fix quick install "" database name.

* Hide Database Configuration section when a connection string is pre-set.

Doesn't seem worth it to extract db name from connection string.

* Ensure wal test 2+

* Fix logging inconsistencies.

* Ensure in transaction when obtaining locks + no-op the SQLite read lock.

There's no point in running the query just to make a single test pass.

* Fix installer database display names

* Allow SQLite shared cache without losing deferred transactions

* Opt into shared cache for new SQLite databases + fix filename

* Fix misc inconsistency in .gitignore

* Prefer our interceptor interface

* Restore DEBUG_DATABASES code OnConnectionOpened in case it's used.

* Back to private cache.

* Added retry strategy for SQLite + refactor out SQL server specific stuff

* Fix SQL server tests.

* Misc - Orphaned comment, incorrect casing.

* InMemory SQLite test database & turn shared cache back on everywhere.

Co-authored-by: Niels Lyngsø <niels.lyngso@gmail.com>
This commit is contained in:
Paul Johnson
2022-03-11 16:14:20 +00:00
committed by GitHub
parent a30d24130f
commit 3961c4c233
201 changed files with 4346 additions and 3564 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
.devcontainer/devcontainer.json
View File

@@ -21,7 +21,7 @@
"profileName": "mssql-container"
}
],
"omnisharp.defaultLaunchSolution": "umbraco-netcore-only.sln",
"omnisharp.defaultLaunchSolution": "umbraco.sln",
"omnisharp.enableDecompilationSupport": true,
"omnisharp.enableRoslynAnalyzers": true
},

2
.github/workflows/codeql-analysis.yml vendored
View File

@@ -28,7 +28,7 @@ jobs:
dotnet-version: '6.0.x'
- name: dotnet build
run: dotnet build umbraco-netcore-only.sln # also runs npm build
run: dotnet build umbraco.sln -c SkipTests
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v1

1
.gitignore vendored
View File

@@ -93,6 +93,7 @@ tests/Umbraco.Tests.AcceptanceTest/cypress/support/chainable.ts
tests/Umbraco.Tests.AcceptanceTest/cypress/videos/
tests/Umbraco.Tests.Integration.SqlCe/DatabaseContextTests.sdf
tests/Umbraco.Tests.Integration.SqlCe/umbraco/Data/TEMP/
tests/Umbraco.Tests.Integration/appsettings.Tests.Local.json
tests/Umbraco.Tests.Integration/TEMP/*
tests/Umbraco.Tests.Integration/umbraco/Data/
tests/Umbraco.Tests.Integration/umbraco/logs/

2
.vscode/tasks.json vendored
View File

@@ -42,7 +42,7 @@
"type": "process",
"args": [
"build",
"${workspaceFolder}/src/umbraco-netcore-only.sln",
"${workspaceFolder}/src/umbraco.sln",
"/property:GenerateFullPaths=true",
"/consoleloggerparameters:NoSummary"
],

43
build/NuSpecs/UmbracoCms.SqlCe.nuspec
View File

@@ -1,43 +0,0 @@
<?xml version="1.0" encoding="utf-8"?>
<package xmlns="http://schemas.microsoft.com/packaging/2010/07/nuspec.xsd">
<metadata minClientVersion="4.1.0">
<id>Umbraco.Cms.SqlCe</id>
<version>9.0.0</version>
<title>Umbraco Cms Sql Ce Add-on</title>
<authors>Umbraco HQ</authors>
<owners>Umbraco HQ</owners>
<license type="expression">MIT</license>
<projectUrl>https://umbraco.com/</projectUrl>
<iconUrl>https://umbraco.com/dist/nuget/logo-small.png</iconUrl>
<requireLicenseAcceptance>false</requireLicenseAcceptance>
<description>Contains the SQL CE assemblies needed to run Umbraco Cms. This package only contains assemblies and can be used for package development. Use the UmbracoCms package to setup Umbraco in Visual Studio as an ASP.NET Core project.</description>
<summary>Contains the SQL CE assemblies needed to run Umbraco Cms</summary>
<language>en-US</language>
<tags>umbraco</tags>
<repository type="git" url="https://github.com/umbraco/umbraco-cms" />
<dependencies>
<group targetFramework="netstandard2.0">
<!--
note: dependencies are specified as [x.y.z,x.999999) eg [2.1.0,2.999999) and NOT [2.1.0,3.0.0) because
the latter would pick anything below 3.0.0 and that includes prereleases such as 3.0.0-alpha, and we do
not want this to happen as the alpha of the next major is, really, the next major already.
-->
<dependency id="Umbraco.Cms.Core" version="[$version$]" />
<dependency id="Umbraco.SqlServerCE" version="[4.0.0.1,4.999999)" /> <!-- Hack it is only available in framework, but we need it on netstandard -->
</group>
</dependencies>
</metadata>
<files>
<!-- libs -->
<file src="$BuildTmp$\SqlCe\Umbraco.Persistence.SqlCe.dll" target="lib\netstandard2.0\Umbraco.Persistence.SqlCe.dll" />
<file src="$BuildTmp$\SqlCe\System.Data.SqlServerCe.dll" target="lib\netstandard2.0\System.Data.SqlServerCe.dll" /> <!-- Hack because the file from the package is only added to net472 projects -->
<!-- docs -->
<file src="$BuildTmp$\SqlCe\Umbraco.Persistence.SqlCe.xml" target="lib\netstandard2.0\Umbraco.Persistence.SqlCe.xml" />
<!-- symbols -->
<file src="$BuildTmp$\SqlCe\Umbraco.Persistence.SqlCe.pdb" target="lib\netstandard2.0\Umbraco.Persistence.SqlCe.pdb" />
</files>
</package>

2
build/NuSpecs/UmbracoCms.nuspec
View File

@@ -20,6 +20,8 @@
<dependency id="Umbraco.Cms.Web.Website" version="[$version$]" />
<dependency id="Umbraco.Cms.Web.BackOffice" version="[$version$]" />
<dependency id="Umbraco.Cms.StaticAssets" version="[$version$]" />
<dependency id="Umbraco.Cms.Persistence.SqlServer" version="[$version$]" />
<dependency id="Umbraco.Cms.Persistence.Sqlite" version="[$version$]" />
</group>
</dependencies>
<!--

252
build/azure-pipelines.yml
View File

@@ -56,7 +56,7 @@ stages:
displayName: dotnet build
inputs:
command: build
projects: '**/umbraco-netcore-only.sln'
projects: '**/umbraco.sln'
- task: DotNetCoreCLI@2
displayName: dotnet test
inputs:
@@ -76,7 +76,7 @@ stages:
displayName: dotnet build
inputs:
command: build
projects: '**/umbraco-netcore-only.sln'
projects: '**/umbraco.sln'
- task: DotNetCoreCLI@2
displayName: dotnet test
inputs:
@@ -103,58 +103,117 @@ stages:
command: test
projects: '**/*.Tests.UnitTests.csproj'
arguments: '--no-build'
- stage: Integration_Tests
displayName: Integration Tests
dependsOn: []
jobs:
- job: Linux_Integration_Tests
services:
mssql: mssql
timeoutInMinutes: 120
displayName: Linux
pool:
vmImage: ubuntu-latest
steps:
- task: UseDotNet@2
displayName: Use .Net 6.x
inputs:
version: 6.x
- task: DotNetCoreCLI@2
displayName: dotnet build
inputs:
command: build
projects: '**/umbraco-netcore-only.sln'
- task: DotNetCoreCLI@2
displayName: dotnet test
inputs:
command: test
projects: '**/Umbraco.Tests.Integration.csproj'
arguments: '--no-build'
env:
UmbracoIntegrationTestConnectionString: 'Server=localhost,1433;User Id=sa;Password=$(SA_PASSWORD);'
- job: Windows_Integration_Tests
timeoutInMinutes: 120
displayName: Windows
pool:
vmImage: windows-latest
steps:
- task: UseDotNet@2
displayName: Use .Net 6.x
inputs:
version: 6.x
- powershell: sqllocaldb start mssqllocaldb
displayName: Start MSSQL LocalDb
- task: DotNetCoreCLI@2
displayName: dotnet build
inputs:
command: build
projects: '**/umbraco.sln'
- task: DotNetCoreCLI@2
displayName: dotnet test
inputs:
command: test
projects: '**\Umbraco.Tests.Integration*.csproj'
arguments: '--no-build'
- job: Linux_Integration_Tests_SQLite
timeoutInMinutes: 120
displayName: Linux (SQLite)
pool:
vmImage: ubuntu-latest
steps:
- task: UseDotNet@2
displayName: Use .Net 6.x
inputs:
version: 6.x
- task: DotNetCoreCLI@2
displayName: dotnet build
inputs:
command: build
projects: '**/umbraco.sln'
- task: DotNetCoreCLI@2
displayName: dotnet test
inputs:
command: test
projects: '**/Umbraco.Tests.Integration.csproj'
arguments: '--no-build'
env:
Tests__Database__DatabaseType: 'Sqlite'
Umbraco__Cms__global__MainDomLock: 'FileSystemMainDomLock'
- job: Linux_Integration_Tests_SQLServer
services:
mssql: mssql
timeoutInMinutes: 120
displayName: Linux (SQL Server)
pool:
vmImage: ubuntu-latest
steps:
- task: UseDotNet@2
displayName: Use .Net 6.x
inputs:
version: 6.x
- task: DotNetCoreCLI@2
displayName: dotnet build
inputs:
command: build
projects: '**/umbraco.sln'
- task: DotNetCoreCLI@2
displayName: dotnet test
inputs:
command: test
projects: '**/Umbraco.Tests.Integration.csproj'
arguments: '--no-build'
env:
Tests__Database__DatabaseType: 'SqlServer'
Tests__Database__SQLServerMasterConnectionString: 'Server=localhost,1433;User Id=sa;Password=$(SA_PASSWORD);'
Umbraco__Cms__global__MainDomLock: 'SqlMainDomLock'
- job: macOS_Integration_Tests_SQLite
timeoutInMinutes: 120
displayName: macOS (SQLite)
pool:
vmImage: macOS-latest
steps:
- task: UseDotNet@2
displayName: Use .Net 6.x
inputs:
version: 6.x
- task: DotNetCoreCLI@2
displayName: dotnet build
inputs:
command: build
projects: '**/umbraco.sln'
- task: DotNetCoreCLI@2
displayName: dotnet test
inputs:
command: test
projects: '**/Umbraco.Tests.Integration.csproj'
arguments: '--no-build'
env:
Tests__Database__DatabaseType: 'Sqlite'
Umbraco__Cms__global__MainDomLock: 'FileSystemMainDomLock'
- job: Windows_Integration_Tests_LocalDb
timeoutInMinutes: 120
displayName: Windows (LocalDb)
pool:
vmImage: windows-latest
steps:
- task: UseDotNet@2
displayName: Use .Net 6.x
inputs:
version: 6.x
- powershell: sqllocaldb start mssqllocaldb
displayName: Start MSSQL LocalDb
- task: DotNetCoreCLI@2
displayName: dotnet build
inputs:
command: build
projects: '**/umbraco.sln'
- task: DotNetCoreCLI@2
displayName: dotnet test
inputs:
command: test
projects: '**/Umbraco.Tests.Integration.csproj'
arguments: '--no-build'
env:
Tests__Database__DatabaseType: 'LocalDb'
Umbraco__Cms__global__MainDomLock: 'MainDomSemaphoreLock'
- stage: Acceptance_Tests
displayName: Acceptance Tests
dependsOn: []
@@ -261,9 +320,10 @@ stages:
displayName: "Publish test artifacts"
inputs:
targetPath: '$(Build.SourcesDirectory)/tests/Umbraco.Tests.AcceptanceTest/cypress/artifacts'
artifact: 'Test artifacts - Windows'
- job: Linux_Acceptance_tests
displayName: Linux
artifact: 'Test artifacts - Windows - Attempt #$(System.JobAttempt)'
- job: Linux_Acceptance_tests_SqlServer
displayName: Linux (SQL Server)
variables:
- name: UmbracoDatabaseServer
value: localhost
@@ -362,12 +422,94 @@ stages:
displayName: "Publish test artifacts"
inputs:
targetPath: '$(Build.SourcesDirectory)/tests/Umbraco.Tests.AcceptanceTest/cypress/artifacts'
artifact: 'Test artifacts - Linux'
artifact: 'Test artifacts - Linux (SQL Server) - Attempt #$(System.JobAttempt)'
- task: PublishPipelineArtifact@1
displayName: "Publish run log"
inputs:
targetPath: '$(Build.ArtifactStagingDirectory)/dotnet_run_log_linux.txt'
artifact: Test Run logs - Linux
artifact: 'Test Run logs - Linux (SQL Server) - Attempt #$(System.JobAttempt)'
- job: Linux_Acceptance_tests_SQLite
displayName: Linux (SQLite)
variables:
- name: ConnectionStrings__umbracoDbDSN
value: Data Source=|DataDirectory|/umbraco-cms-cypress.sqlite.db;Cache=Private;Foreign Keys=True
- name: ConnectionStrings__umbracoDbDSN_ProviderName
value: Microsoft.Data.SQLite
pool:
vmImage: ubuntu-latest
steps:
- task: UseDotNet@2
displayName: Use .Net 6.x
inputs:
version: 6.x
- task: NodeTool@0
displayName: Use Node $(nodeVersion)
inputs:
versionSpec: $(nodeVersion)
- task: Npm@1
displayName: npm ci (Client)
inputs:
command: ci
workingDir: src/Umbraco.Web.UI.Client
verbose: false
- task: gulp@0
displayName: gulp build
inputs:
gulpFile: src/Umbraco.Web.UI.Client/gulpfile.js
targets: build
workingDirectory: src/Umbraco.Web.UI.Client
- task: DotNetCoreCLI@2
displayName: dotnet build
inputs:
command: build
projects: src/Umbraco.Web.UI/Umbraco.Web.UI.csproj
- task: Bash@3
displayName: dotnet run
inputs:
targetType: 'inline'
script: 'nohup dotnet run --no-build -p ./src/Umbraco.Web.UI/ > $(Build.ArtifactStagingDirectory)/dotnet_run_log_linux.txt &'
- task: Bash@3
displayName: Generate Cypress.env.json
inputs:
targetType: 'inline'
script: 'echo "{ \"username\": \"$USERNAME\", \"password\": \"$PASSWORD\" }" > "tests/Umbraco.Tests.AcceptanceTest/cypress.env.json"'
env:
USERNAME: $(Umbraco__CMS__Unattended__UnattendedUserEmail)
PASSWORD: $(Umbraco__CMS__Unattended__UnattendedUserPassword)
- task: Npm@1
name: PrepareTask
displayName: npm ci (AcceptanceTest)
inputs:
command: ci
workingDir: 'tests/Umbraco.Tests.AcceptanceTest'
- task: Bash@3
displayName: Run Cypress (Desktop)
condition: always()
continueOnError: true
inputs:
targetType: inline
workingDirectory: tests/Umbraco.Tests.AcceptanceTest
script: 'npm run test -- --reporter junit --reporter-options "mochaFile=results/test-output-D-[hash].xml,toConsole=true" --config="viewportHeight=1600,viewportWidth=2560,screenshotsFolder=cypress/artifacts/desktop/screenshots,videosFolder=cypress/artifacts/desktop/videos,videoUploadOnPasses=false"'
- task: PublishTestResults@2
condition: always()
inputs:
testResultsFormat: 'JUnit'
testResultsFiles: 'tests/Umbraco.Tests.AcceptanceTest/results/test-output-D-*.xml'
mergeTestResults: true
testRunTitle: "Test results Desktop"
- task: PublishPipelineArtifact@1
displayName: "Publish test artifacts"
inputs:
targetPath: '$(Build.SourcesDirectory)/tests/Umbraco.Tests.AcceptanceTest/cypress/artifacts'
artifact: 'Test artifacts - Linux (SQLite) - Attempt #$(System.JobAttempt)'
- task: PublishPipelineArtifact@1
displayName: "Publish run log"
inputs:
targetPath: '$(Build.ArtifactStagingDirectory)/dotnet_run_log_linux.txt'
artifact: 'Test Run logs - Linux (SQLite) - Attempt #$(System.JobAttempt)'
- stage: Artifacts
dependsOn: []
jobs:

33
build/build.ps1
View File

@@ -189,10 +189,6 @@
--configuration Release --output "$($this.BuildTemp)\WebApp\bin\\" `
> $log
& dotnet publish "$src\Umbraco.Persistence.SqlCe\Umbraco.Persistence.SqlCe.csproj" `
--configuration Release --output "$($this.BuildTemp)\SqlCe\" `
> $log
# remove extra files
$webAppBin = "$($this.BuildTemp)\WebApp\bin"
$excludeDirs = @("$($webAppBin)\refs","$($webAppBin)\runtimes","$($webAppBin)\Umbraco","$($webAppBin)\wwwroot")
@@ -278,9 +274,6 @@
/p:UmbracoBuild=True `
> $log
# copy Umbraco.Persistence.SqlCe files into WebApp
Copy-Item "$($this.BuildTemp)\tests\Umbraco.Persistence.SqlCe.*" "$($this.BuildTemp)\WebApp\bin"
if (-not $?) { throw "Failed to compile tests." }
# /p:UmbracoBuild tells the csproj that we are building from PS
@@ -325,18 +318,6 @@
$_.LastWriteTime = $_.LastWriteTime.AddHours(-11)
}
# copy libs
Write-Host "Copy SqlCE libraries"
$nugetPackages = $env:NUGET_PACKAGES
if (-not $nugetPackages)
{
$nugetPackages = [System.Environment]::ExpandEnvironmentVariables("%userprofile%\.nuget\packages")
}
#$this.CopyFiles("$nugetPackages\umbraco.sqlserverce\4.0.0.1\runtimes\win-x86\native", "*.*", "$tmp\bin\x86")
#$this.CopyFiles("$nugetPackages\umbraco.sqlserverce\4.0.0.1\runtimes\win-x64\native", "*.*", "$tmp\bin\amd64")
#$this.CopyFiles("$nugetPackages\umbraco.sqlserverce\4.0.0.1\runtimes\win-x86\native", "*.*", "$tmp\WebApp\bin\x86")
#$this.CopyFiles("$nugetPackages\umbraco.sqlserverce\4.0.0.1\runtimes\win-x64\native", "*.*", "$tmp\WebApp\bin\amd64")
# copy Belle
Write-Host "Copy Belle"
$this.CopyFiles("$src\Umbraco.Web.UI\wwwroot\umbraco\assets", "*", "$tmp\WebApp\wwwroot\umbraco\assets")
@@ -344,8 +325,6 @@
$this.CopyFiles("$src\Umbraco.Web.UI\wwwroot\umbraco\lib", "*", "$tmp\WebApp\wwwroot\umbraco\lib")
$this.CopyFiles("$src\Umbraco.Web.UI\wwwroot\umbraco\views", "*", "$tmp\WebApp\wwwroot\umbraco\views")
# Prepare templates
Write-Host "Copy template files"
$this.CopyFiles("$templates", "*", "$tmp\Templates")
@@ -390,7 +369,7 @@
Write-Host "Restore NuGet"
Write-Host "Logging to $($this.BuildTemp)\nuget.restore.log"
$params = "-Source", $nugetsourceUmbraco
&$this.BuildEnv.NuGet restore "$($this.SolutionRoot)\umbraco-netcore-only.sln" > "$($this.BuildTemp)\nuget.restore.log" @params
&$this.BuildEnv.NuGet restore "$($this.SolutionRoot)\umbraco.sln" > "$($this.BuildTemp)\nuget.restore.log" @params
if (-not $?) { throw "Failed to restore NuGet packages." }
})
@@ -401,7 +380,7 @@
Write-Host "Create NuGet packages"
&dotnet pack "$($this.SolutionRoot)\umbraco-netcore-only.sln" `
&dotnet pack "$($this.SolutionRoot)\umbraco.sln" `
--output "$($this.BuildOutput)" `
--verbosity detailed `
-c Release `
@@ -413,12 +392,6 @@
-Verbosity detailed -outputDirectory "$($this.BuildOutput)" > "$($this.BuildTemp)\nupack.cms.log"
if (-not $?) { throw "Failed to pack NuGet UmbracoCms." }
&$this.BuildEnv.NuGet Pack "$nuspecs\UmbracoCms.SqlCe.nuspec" `
-Properties BuildTmp="$($this.BuildTemp)" `
-Version "$($this.Version.Semver.ToString())" `
-Verbosity detailed -outputDirectory "$($this.BuildOutput)" > "$($this.BuildTemp)\nupack.cmssqlce.log"
if (-not $?) { throw "Failed to pack NuGet UmbracoCms.SqlCe." }
&$this.BuildEnv.NuGet Pack "$nuspecs\UmbracoCms.StaticAssets.nuspec" `
-Properties BuildTmp="$($this.BuildTemp)" `
-Version "$($this.Version.Semver.ToString())" `
@@ -445,7 +418,7 @@
{
$this.VerifyNuGetConsistency(
("UmbracoCms"),
("Umbraco.Core", "Umbraco.Infrastructure", "Umbraco.Web.UI", "Umbraco.Examine.Lucene", "Umbraco.PublishedCache.NuCache", "Umbraco.Web.Common", "Umbraco.Web.Website", "Umbraco.Web.BackOffice", "Umbraco.Persistence.SqlCe"))
("Umbraco.Core", "Umbraco.Infrastructure", "Umbraco.Web.UI", "Umbraco.Examine.Lucene", "Umbraco.PublishedCache.NuCache", "Umbraco.Web.Common", "Umbraco.Web.Website", "Umbraco.Web.BackOffice", "Umbraco.Cms.Persistence.Sqlite", "Umbraco.Cms.Persistence.SqlServer"))
if ($this.OnError()) { return }
})

5
build/templates/UmbracoProject/.template.config/dotnetcli.host.json
View File

@@ -5,10 +5,6 @@
"longName": "PackageTestSiteName",
"shortName": "p"
},
"UseSqlCe": {
"longName": "SqlCe",
"shortName": "ce"
},
"SkipRestore": {
"longName": "no-restore",
"shortName": ""
@@ -40,7 +36,6 @@
},
"usageExamples": [
"dotnet new umbraco -n MyNewProject",
"dotnet new umbraco -n MyNewProjectWithCE -ce",
"dotnet new umbraco -n MyNewProject --no-restore",
"dotnet new umbraco -n MyNewProject --friendly-name \"Friendly User\" --email user@email.com --password password1234 --connection-string \"Server=ConnectionStringHere\""
]

7
build/templates/UmbracoProject/.template.config/ide.host.json
View File

@@ -7,13 +7,6 @@
"text": "Umbraco Web Application - An empty Umbraco CMS web application"
},
"symbolInfo": [
{
"id": "UseSqlCe",
"name": {
"text": "Use Sql Compact Edition (SqlCE)"
},
"isVisible": "true"
},
{
"id": "SkipRestore",
"name": {

6
build/templates/UmbracoProject/.template.config/template.json
View File

@@ -68,12 +68,6 @@
"replaces":"PackageTestSiteName",
"description": "The name of the package this should be a test site for (Default: '')"
},
"UseSqlCe":{
"type": "parameter",
"datatype":"bool",
"defaultValue": "false",
"description": "Adds the required dependencies to use SqlCE (Windows only) (Default: false)"
},
"Framework": {
"type": "parameter",
"description": "The target framework for the project",

2
build/templates/UmbracoProject/UmbracoProject.csproj
View File

@@ -7,8 +7,6 @@
<ItemGroup>
<PackageReference Include="Umbraco.Cms" Version="UMBRACO_VERSION_FROM_TEMPLATE" />
<PackageReference Include="Umbraco.Cms.SqlCe" Version="UMBRACO_VERSION_FROM_TEMPLATE" Condition="'$(UseSqlCe)' == 'true'" />
<PackageReference Include="Umbraco.SqlServerCE" Version="4.0.0.1" Condition="'$(UseSqlCe)' == 'true'" />
</ItemGroup>
<!-- Force windows to use ICU. Otherwise Windows 10 2019H1+ will do it, but older windows 10 and most if not all winodws servers will run NLS -->

12
src/Umbraco.Cms.Persistence.SqlServer/Constants.cs Normal file
View File

@@ -0,0 +1,12 @@
namespace Umbraco.Cms.Persistence.SqlServer;
/// <summary>
/// Constants related to SQLite.
/// </summary>
public static class Constants
{
/// <summary>
/// SQLite provider name.
/// </summary>
public const string ProviderName = "Microsoft.Data.SqlClient";
}

2
src/Umbraco.Infrastructure/Persistence/Dtos/ColumnInSchemaDto.cs → src/Umbraco.Cms.Persistence.SqlServer/Dtos/ColumnInSchemaDto.cs
View File

@@ -1,6 +1,6 @@
using NPoco;
namespace Umbraco.Cms.Infrastructure.Persistence.Dtos
namespace Umbraco.Cms.Persistence.SqlServer.Dtos
{
internal class ColumnInSchemaDto
{

2
src/Umbraco.Infrastructure/Persistence/Dtos/ConstraintPerColumnDto.cs → src/Umbraco.Cms.Persistence.SqlServer/Dtos/ConstraintPerColumnDto.cs
View File

@@ -1,6 +1,6 @@
using NPoco;
namespace Umbraco.Cms.Infrastructure.Persistence.Dtos
namespace Umbraco.Cms.Persistence.SqlServer.Dtos
{
internal class ConstraintPerColumnDto
{

2
src/Umbraco.Infrastructure/Persistence/Dtos/ConstraintPerTableDto.cs → src/Umbraco.Cms.Persistence.SqlServer/Dtos/ConstraintPerTableDto.cs
View File

@@ -1,6 +1,6 @@
using NPoco;
namespace Umbraco.Cms.Infrastructure.Persistence.Dtos
namespace Umbraco.Cms.Persistence.SqlServer.Dtos
{
internal class ConstraintPerTableDto
{

2
src/Umbraco.Infrastructure/Persistence/Dtos/DefaultConstraintPerColumnDto.cs → src/Umbraco.Cms.Persistence.SqlServer/Dtos/DefaultConstraintPerColumnDto.cs
View File

@@ -1,6 +1,6 @@
using NPoco;
namespace Umbraco.Cms.Infrastructure.Persistence.Dtos
namespace Umbraco.Cms.Persistence.SqlServer.Dtos
{
internal class DefaultConstraintPerColumnDto
{

2
src/Umbraco.Infrastructure/Persistence/Dtos/DefinedIndexDto.cs → src/Umbraco.Cms.Persistence.SqlServer/Dtos/DefinedIndexDto.cs
View File

@@ -1,6 +1,6 @@
using NPoco;
namespace Umbraco.Cms.Infrastructure.Persistence.Dtos
namespace Umbraco.Cms.Persistence.SqlServer.Dtos
{
internal class DefinedIndexDto
{

11
src/Umbraco.Cms.Persistence.SqlServer/Interceptors/SqlServerAddMiniProfilerInterceptor.cs Normal file
View File

@@ -0,0 +1,11 @@
using System.Data.Common;
using NPoco;
using StackExchange.Profiling;
namespace Umbraco.Cms.Persistence.SqlServer.Interceptors;
public class SqlServerAddMiniProfilerInterceptor : SqlServerConnectionInterceptor
{
public override DbConnection OnConnectionOpened(IDatabase database, DbConnection conn)
=> new StackExchange.Profiling.Data.ProfiledDbConnection(conn, MiniProfiler.Current);
}

34
src/Umbraco.Cms.Persistence.SqlServer/Interceptors/SqlServerAddRetryPolicyInterceptor.cs Normal file
View File

@@ -0,0 +1,34 @@
using System.Data.Common;
using Microsoft.Extensions.Options;
using NPoco;
using Umbraco.Cms.Core.Configuration.Models;
using Umbraco.Cms.Infrastructure.Persistence.FaultHandling;
using Umbraco.Extensions;
namespace Umbraco.Cms.Persistence.SqlServer.Interceptors;
public class SqlServerAddRetryPolicyInterceptor : SqlServerConnectionInterceptor
{
private readonly IOptionsMonitor<ConnectionStrings> _connectionStrings;
public SqlServerAddRetryPolicyInterceptor(IOptionsMonitor<ConnectionStrings> connectionStrings)
=> _connectionStrings = connectionStrings;
public override DbConnection OnConnectionOpened(IDatabase database, DbConnection conn)
{
if (!_connectionStrings.CurrentValue.IsConnectionStringConfigured())
{
return conn;
}
RetryPolicy? connectionRetryPolicy = RetryPolicyFactory.GetDefaultSqlConnectionRetryPolicyByConnectionString(_connectionStrings.CurrentValue.ConnectionString);
RetryPolicy? commandRetryPolicy = RetryPolicyFactory.GetDefaultSqlCommandRetryPolicyByConnectionString(_connectionStrings.CurrentValue.ConnectionString);
if (connectionRetryPolicy == null && commandRetryPolicy == null)
{
return conn;
}
return new RetryDbConnection(conn, connectionRetryPolicy, commandRetryPolicy);
}
}

16
src/Umbraco.Cms.Persistence.SqlServer/Interceptors/SqlServerConnectionInterceptor.cs Normal file
View File

@@ -0,0 +1,16 @@
using System.Data.Common;
using NPoco;
using Umbraco.Cms.Infrastructure.Persistence;
namespace Umbraco.Cms.Persistence.SqlServer.Interceptors;
public abstract class SqlServerConnectionInterceptor : IProviderSpecificConnectionInterceptor
{
public string ProviderName => Constants.ProviderName;
public abstract DbConnection OnConnectionOpened(IDatabase database, DbConnection conn);
public virtual void OnConnectionClosing(IDatabase database, DbConnection conn)
{
}
}

4
src/Umbraco.Infrastructure/Persistence/BulkDataReader.cs → src/Umbraco.Cms.Persistence.SqlServer/Services/BulkDataReader.cs
View File

@@ -1,5 +1,3 @@
using System;
using System.Collections.Generic;
using System.Collections.ObjectModel;
using System.Data;
using System.Data.Common;
@@ -7,7 +5,7 @@ using System.Diagnostics;
using System.Globalization;
using Microsoft.Data.SqlClient;
namespace Umbraco.Cms.Infrastructure.Persistence
namespace Umbraco.Cms.Persistence.SqlServer.Services
{
/// <summary>
/// A base implementation of <see cref="IDataReader"/> that is suitable for <see cref="SqlBulkCopy.WriteToServer(IDataReader)"/>.

52
src/Umbraco.Infrastructure/Persistence/SqlSyntax/MicrosoftSqlSyntaxProviderBase.cs → src/Umbraco.Cms.Persistence.SqlServer/Services/MicrosoftSqlSyntaxProviderBase.cs
View File

@@ -1,11 +1,12 @@
using System;
using System.Data;
using System.Linq;
using Microsoft.Extensions.Logging;
using NPoco;
using Umbraco.Cms.Core;
using Umbraco.Cms.Core.Persistence;
using Umbraco.Cms.Infrastructure.Persistence.Querying;
using Umbraco.Cms.Infrastructure.Persistence.DatabaseModelDefinitions;
using Umbraco.Cms.Infrastructure.Persistence.SqlSyntax;
namespace Umbraco.Cms.Infrastructure.Persistence.SqlSyntax
namespace Umbraco.Cms.Persistence.SqlServer.Services
{
/// <summary>
/// Abstract class for defining MS sql implementations
@@ -14,8 +15,12 @@ namespace Umbraco.Cms.Infrastructure.Persistence.SqlSyntax
public abstract class MicrosoftSqlSyntaxProviderBase<TSyntax> : SqlSyntaxProviderBase<TSyntax>
where TSyntax : ISqlSyntaxProvider
{
private readonly ILogger _logger;
protected MicrosoftSqlSyntaxProviderBase()
{
_logger = StaticApplicationLogging.CreateLogger<TSyntax>();
AutoIncrementDefinition = "IDENTITY(1,1)";
GuidColumnDefinition = "UniqueIdentifier";
RealColumnDefinition = "FLOAT";
@@ -34,7 +39,7 @@ namespace Umbraco.Cms.Infrastructure.Persistence.SqlSyntax
if (tableName.Contains(".") == false)
return $"[{tableName}]";
var tableNameParts = tableName.Split(Constants.CharArrays.Period, 2);
var tableNameParts = tableName.Split(Cms.Core.Constants.CharArrays.Period, 2);
return $"[{tableNameParts[0]}].[{tableNameParts[1]}]";
}
@@ -175,5 +180,42 @@ namespace Umbraco.Cms.Infrastructure.Persistence.SqlSyntax
}
return sqlDbType;
}
public override void HandleCreateTable(IDatabase database, TableDefinition tableDefinition, bool skipKeysAndIndexes = false)
{
var createSql = Format(tableDefinition);
var createPrimaryKeySql = FormatPrimaryKey(tableDefinition);
List<string> foreignSql = Format(tableDefinition.ForeignKeys);
_logger.LogInformation("Create table:\n {Sql}", createSql);
database.Execute(new Sql(createSql));
if (skipKeysAndIndexes)
{
return;
}
//If any statements exists for the primary key execute them here
if (string.IsNullOrEmpty(createPrimaryKeySql) == false)
{
_logger.LogInformation("Create Primary Key:\n {Sql}", createPrimaryKeySql);
database.Execute(new Sql(createPrimaryKeySql));
}
List<string> indexSql = Format(tableDefinition.Indexes);
//Loop through index statements and execute sql
foreach (var sql in indexSql)
{
_logger.LogInformation("Create Index:\n {Sql}", sql);
database.Execute(new Sql(sql));
}
//Loop through foreignkey statements and execute sql
foreach (var sql in foreignSql)
{
_logger.LogInformation("Create Foreign Key:\n {Sql}", sql);
database.Execute(new Sql(sql));
}
}
}
}

6
src/Umbraco.Infrastructure/Persistence/PocoDataDataReader.cs → src/Umbraco.Cms.Persistence.SqlServer/Services/PocoDataDataReader.cs
View File

@@ -1,14 +1,10 @@
using System;
using System.Collections.Generic;
using System.Data;
using System.Linq;
using NPoco;
using Umbraco.Cms.Infrastructure.Persistence.DatabaseAnnotations;
using Umbraco.Cms.Infrastructure.Persistence.DatabaseModelDefinitions;
using Umbraco.Cms.Infrastructure.Persistence.SqlSyntax;
using Umbraco.Extensions;
namespace Umbraco.Cms.Infrastructure.Persistence
namespace Umbraco.Cms.Persistence.SqlServer.Services
{
/// <summary>
/// A data reader used for reading collections of PocoData entity types

101
src/Umbraco.Cms.Persistence.SqlServer/Services/SqlAzureDatabaseProviderMetadata.cs Normal file
View File

@@ -0,0 +1,101 @@
using System.Runtime.Serialization;
using Umbraco.Cms.Core.Install.Models;
using Umbraco.Cms.Infrastructure.Persistence;
using Umbraco.Extensions;
namespace Umbraco.Cms.Persistence.SqlServer.Services;
/// <summary>
/// Provider metadata for SQL Azure
/// </summary>
[DataContract]
public class SqlAzureDatabaseProviderMetadata : IDatabaseProviderMetadata
{
/// <inheritdoc />
public Guid Id => new ("7858e827-8951-4fe0-a7fe-6883011b1f1b");
/// <inheritdoc />
public int SortOrder => 3;
/// <inheritdoc />
public string DisplayName => "Azure SQL";
/// <inheritdoc />
public string DefaultDatabaseName => string.Empty;
/// <inheritdoc />
public string ProviderName => Constants.ProviderName;
/// <inheritdoc />
public bool SupportsQuickInstall => false;
/// <inheritdoc />
public bool IsAvailable => true;
/// <inheritdoc />
public bool RequiresServer => true;
/// <inheritdoc />
public string ServerPlaceholder => "umbraco-database.database.windows.net";
/// <inheritdoc />
public bool RequiresCredentials => true;
/// <inheritdoc />
public bool SupportsIntegratedAuthentication => false;
/// <inheritdoc />
public bool RequiresConnectionTest => true;
/// <inheritdoc />
public bool ForceCreateDatabase => false;
/// <inheritdoc />
public string GenerateConnectionString(DatabaseModel databaseModel)
{
var server = databaseModel.Server;
var databaseName = databaseModel.DatabaseName;
var user = databaseModel.Login;
var password = databaseModel.Password;
if (server.Contains(".") && ServerStartsWithTcp(server) == false)
server = $"tcp:{server}";
if (server.Contains(".") == false && ServerStartsWithTcp(server))
{
string serverName = server.Contains(",")
? server.Substring(0, server.IndexOf(",", StringComparison.Ordinal))
: server;
var portAddition = string.Empty;
if (server.Contains(","))
portAddition = server.Substring(server.IndexOf(",", StringComparison.Ordinal));
server = $"{serverName}.database.windows.net{portAddition}";
}
if (ServerStartsWithTcp(server) == false)
server = $"tcp:{server}.database.windows.net";
if (server.Contains(",") == false)
server = $"{server},1433";
if (user.Contains("@") == false)
{
var userDomain = server;
if (ServerStartsWithTcp(server))
userDomain = userDomain.Substring(userDomain.IndexOf(":", StringComparison.Ordinal) + 1);
if (userDomain.Contains("."))
userDomain = userDomain.Substring(0, userDomain.IndexOf(".", StringComparison.Ordinal));
user = $"{user}@{userDomain}";
}
return $"Server={server};Database={databaseName};User ID={user};Password={password}";
}
private static bool ServerStartsWithTcp(string server) => server.InvariantStartsWith("tcp:");
}

66
src/Umbraco.Cms.Persistence.SqlServer/Services/SqlLocalDbDatabaseProviderMetadata.cs Normal file
View File

@@ -0,0 +1,66 @@
using System.Runtime.Serialization;
using Microsoft.Data.SqlClient;
using Umbraco.Cms.Core.Configuration.Models;
using Umbraco.Cms.Core.Install.Models;
using Umbraco.Cms.Infrastructure.Persistence;
namespace Umbraco.Cms.Persistence.SqlServer.Services;
/// <summary>
/// Provider metadata for SQL Server LocalDb
/// </summary>
[DataContract]
public class SqlLocalDbDatabaseProviderMetadata : IDatabaseProviderMetadata
{
/// <inheritdoc />
public Guid Id => new ("05a7e9ed-aa6a-43af-a309-63422c87c675");
/// <inheritdoc />
public int SortOrder => 1;
/// <inheritdoc />
public string DisplayName => "SQL Server Express LocalDB";
/// <inheritdoc />
public string DefaultDatabaseName => Core.Constants.System.UmbracoDefaultDatabaseName;
/// <inheritdoc />
public string ProviderName => Constants.ProviderName;
/// <inheritdoc />
public bool SupportsQuickInstall => true;
/// <inheritdoc />
public bool IsAvailable => new LocalDb().IsAvailable;
/// <inheritdoc />
public bool RequiresServer => false;
/// <inheritdoc />
public string ServerPlaceholder => null;
/// <inheritdoc />
public bool RequiresCredentials => false;
/// <inheritdoc />
public bool SupportsIntegratedAuthentication => false;
/// <inheritdoc />
public bool RequiresConnectionTest => false;
/// <inheritdoc />
public bool ForceCreateDatabase => true;
/// <inheritdoc />
public string GenerateConnectionString(DatabaseModel databaseModel)
{
var builder = new SqlConnectionStringBuilder
{
DataSource = @"(localdb)\MSSQLLocalDB",
AttachDBFilename = @$"{ConnectionStrings.DataDirectoryPlaceholder}\{databaseModel.DatabaseName}.mdf",
IntegratedSecurity = true
};
return builder.ConnectionString;
}
}

13
src/Umbraco.Infrastructure/Persistence/SqlServerBulkSqlInsertProvider.cs → src/Umbraco.Cms.Persistence.SqlServer/Services/SqlServerBulkSqlInsertProvider.cs
View File

@@ -1,20 +1,17 @@
using System;
using System.Collections.Generic;
using System.Data;
using System.Linq;
using Microsoft.Data.SqlClient;
using NPoco;
using Umbraco.Cms.Infrastructure.Persistence.SqlSyntax;
using Umbraco.Cms.Infrastructure.Persistence;
using Umbraco.Extensions;
namespace Umbraco.Cms.Infrastructure.Persistence
namespace Umbraco.Cms.Persistence.SqlServer.Services
{
/// <summary>
/// A bulk sql insert provider for Sql Server
/// </summary>
public class SqlServerBulkSqlInsertProvider : IBulkSqlInsertProvider
{
public string ProviderName => Cms.Core.Constants.DatabaseProviders.SqlServer;
public string ProviderName => Constants.ProviderName;
public int BulkInsertRecords<T>(IUmbracoDatabase database, IEnumerable<T> records)
{
@@ -24,9 +21,7 @@ namespace Umbraco.Cms.Infrastructure.Persistence
var pocoData = database.PocoDataFactory.ForType(typeof(T));
if (pocoData == null) throw new InvalidOperationException("Could not find PocoData for " + typeof(T));
return database.DatabaseType.IsSqlServer2008OrLater()
? BulkInsertRecordsSqlServer(database, pocoData, recordsA)
: BasicBulkSqlInsertProvider.BulkInsertRecordsWithCommands(database, recordsA);
return BulkInsertRecordsSqlServer(database, pocoData, recordsA);
}
/// <summary>

8
src/Umbraco.Infrastructure/Persistence/SqlServerDatabaseCreator.cs → src/Umbraco.Cms.Persistence.SqlServer/Services/SqlServerDatabaseCreator.cs
View File

@@ -1,13 +1,11 @@
using System;
using System.IO;
using Microsoft.Data.SqlClient;
using Umbraco.Cms.Core;
using Umbraco.Cms.Infrastructure.Persistence;
namespace Umbraco.Cms.Infrastructure.Persistence
namespace Umbraco.Cms.Persistence.SqlServer.Services
{
public class SqlServerDatabaseCreator : IDatabaseCreator
{
public string ProviderName => Constants.DatabaseProviders.SqlServer;
public string ProviderName => Constants.ProviderName;
public void Create(string connectionString)
{

57
src/Umbraco.Cms.Persistence.SqlServer/Services/SqlServerDatabaseProviderMetadata.cs Normal file
View File

@@ -0,0 +1,57 @@
using System.Runtime.Serialization;
using Umbraco.Cms.Core.Install.Models;
using Umbraco.Cms.Infrastructure.Persistence;
namespace Umbraco.Cms.Persistence.SqlServer.Services;
/// <summary>
/// Provider metadata for SQL Server
/// </summary>
[DataContract]
public class SqlServerDatabaseProviderMetadata : IDatabaseProviderMetadata
{
/// <inheritdoc />
public Guid Id => new ("5e1ad149-1951-4b74-90bf-2ac2aada9e73");
/// <inheritdoc />
public int SortOrder => 2;
/// <inheritdoc />
public string DisplayName => "SQL Server";
/// <inheritdoc />
public string DefaultDatabaseName => string.Empty;
/// <inheritdoc />
public string ProviderName => Constants.ProviderName;
/// <inheritdoc />
public bool SupportsQuickInstall => false;
/// <inheritdoc />
public bool IsAvailable => true;
/// <inheritdoc />
public bool RequiresServer => true;
/// <inheritdoc />
public string ServerPlaceholder => "(local)\\SQLEXPRESS";
/// <inheritdoc />
public bool RequiresCredentials => true;
/// <inheritdoc />
public bool SupportsIntegratedAuthentication => true;
/// <inheritdoc />
public bool RequiresConnectionTest => true;
/// <inheritdoc />
public bool ForceCreateDatabase => false;
/// <inheritdoc />
public string GenerateConnectionString(DatabaseModel databaseModel) =>
databaseModel.IntegratedAuth
? $"Server={databaseModel.Server};Database={databaseModel.DatabaseName};Integrated Security=true"
: $"Server={databaseModel.Server};Database={databaseModel.DatabaseName};User Id={databaseModel.Login};Password={databaseModel.Password}";
}

169
src/Umbraco.Cms.Persistence.SqlServer/Services/SqlServerDistributedLockingMechanism.cs Normal file
View File

@@ -0,0 +1,169 @@
using System.Data;
using Microsoft.Data.SqlClient;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Options;
using Umbraco.Cms.Core.Configuration.Models;
using Umbraco.Cms.Core.DistributedLocking;
using Umbraco.Cms.Core.DistributedLocking.Exceptions;
using Umbraco.Cms.Infrastructure.Persistence;
using Umbraco.Cms.Infrastructure.Scoping;
using Umbraco.Extensions;
namespace Umbraco.Cms.Persistence.SqlServer.Services;
/// <summary>
/// SQL Server implementation of <see cref="IDistributedLockingMechanism"/>.
/// </summary>
public class SqlServerDistributedLockingMechanism : IDistributedLockingMechanism
{
private readonly ILogger<SqlServerDistributedLockingMechanism> _logger;
private readonly Lazy<IScopeAccessor> _scopeAccessor; // Hooray it's a circular dependency.
private readonly IOptionsMonitor<GlobalSettings> _globalSettings;
private readonly IOptionsMonitor<ConnectionStrings> _connectionStrings;
/// <summary>
/// Initializes a new instance of the <see cref="SqlServerDistributedLockingMechanism"/> class.
/// </summary>
public SqlServerDistributedLockingMechanism(
ILogger<SqlServerDistributedLockingMechanism> logger,
Lazy<IScopeAccessor> scopeAccessor,
IOptionsMonitor<GlobalSettings> globalSettings,
IOptionsMonitor<ConnectionStrings> connectionStrings)
{
_logger = logger;
_scopeAccessor = scopeAccessor;
_globalSettings = globalSettings;
_connectionStrings = connectionStrings;
}
/// <inheritdoc />
public bool Enabled => _connectionStrings.CurrentValue.IsConnectionStringConfigured() &&
_connectionStrings.CurrentValue.ProviderName == Constants.ProviderName;
/// <inheritdoc />
public IDistributedLock ReadLock(int lockId, TimeSpan? obtainLockTimeout = null)
{
obtainLockTimeout ??= _globalSettings.CurrentValue.DistributedLockingReadLockDefaultTimeout;
return new SqlServerDistributedLock(this, lockId, DistributedLockType.ReadLock, obtainLockTimeout.Value);
}
/// <inheritdoc />
public IDistributedLock WriteLock(int lockId, TimeSpan? obtainLockTimeout = null)
{
obtainLockTimeout ??= _globalSettings.CurrentValue.DistributedLockingWriteLockDefaultTimeout;
return new SqlServerDistributedLock(this, lockId, DistributedLockType.WriteLock, obtainLockTimeout.Value);
}
private class SqlServerDistributedLock : IDistributedLock
{
private readonly SqlServerDistributedLockingMechanism _parent;
private readonly TimeSpan _timeout;
public SqlServerDistributedLock(
SqlServerDistributedLockingMechanism parent,
int lockId,
DistributedLockType lockType,
TimeSpan timeout)
{
_parent = parent;
_timeout = timeout;
LockId = lockId;
LockType = lockType;
_parent._logger.LogDebug("Requesting {lockType} for id {id}", LockType, LockId);
try
{
switch (lockType)
{
case DistributedLockType.ReadLock:
ObtainReadLock();
break;
case DistributedLockType.WriteLock:
ObtainWriteLock();
break;
default:
throw new ArgumentOutOfRangeException(nameof(lockType), lockType, @"Unsupported lockType");
}
}
catch (SqlException ex) when (ex.Number == 1222)
{
if (LockType == DistributedLockType.ReadLock)
{
throw new DistributedReadLockTimeoutException(LockId);
}
throw new DistributedWriteLockTimeoutException(LockId);
}
_parent._logger.LogDebug("Acquired {lockType} for id {id}", LockType, LockId);
}
public int LockId { get; }
public DistributedLockType LockType { get; }
public void Dispose()
{
// Mostly no op, cleaned up by completing transaction in scope.
_parent._logger.LogDebug("Dropped {lockType} for id {id}", LockType, LockId);
}
public override string ToString()
=> $"SqlServerDistributedLock({LockId}, {LockType}";
private void ObtainReadLock()
{
IUmbracoDatabase db = _parent._scopeAccessor.Value.AmbientScope.Database;
if (!db.InTransaction)
{
throw new InvalidOperationException("SqlServerDistributedLockingMechanism requires a transaction to function.");
}
if (db.Transaction.IsolationLevel < IsolationLevel.ReadCommitted)
{
throw new InvalidOperationException("A transaction with minimum ReadCommitted isolation level is required.");
}
const string query = "SELECT value FROM umbracoLock WITH (REPEATABLEREAD) WHERE id=@id";
db.Execute("SET LOCK_TIMEOUT " + _timeout.TotalMilliseconds + ";");
var i = db.ExecuteScalar<int?>(query, new {id = LockId});
if (i == null)
{
// ensure we are actually locking!
throw new ArgumentException(@$"LockObject with id={LockId} does not exist.", nameof(LockId));
}
}
private void ObtainWriteLock()
{
IUmbracoDatabase db = _parent._scopeAccessor.Value.AmbientScope.Database;
if (!db.InTransaction)
{
throw new InvalidOperationException("SqlServerDistributedLockingMechanism requires a transaction to function.");
}
if (db.Transaction.IsolationLevel < IsolationLevel.ReadCommitted)
{
throw new InvalidOperationException("A transaction with minimum ReadCommitted isolation level is required.");
}
const string query = @"UPDATE umbracoLock WITH (REPEATABLEREAD) SET value = (CASE WHEN (value=1) THEN -1 ELSE 1 END) WHERE id=@id";
db.Execute("SET LOCK_TIMEOUT " + _timeout.TotalMilliseconds + ";");
var i = db.Execute(query, new {id = LockId});
if (i == 0)
{
// ensure we are actually locking!
throw new ArgumentException($"LockObject with id={LockId} does not exist.");
}
}
}
}

226
src/Umbraco.Infrastructure/Persistence/SqlSyntax/SqlServerSyntaxProvider.cs → src/Umbraco.Cms.Persistence.SqlServer/Services/SqlServerSyntaxProvider.cs
View File

@@ -1,18 +1,17 @@
using System;
using System.Collections.Generic;
using System.Data;
using System.Linq;
using Microsoft.Data.SqlClient;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Options;
using NPoco;
using Umbraco.Cms.Core;
using Umbraco.Cms.Core.Configuration.Models;
using Umbraco.Cms.Infrastructure.Persistence;
using Umbraco.Cms.Infrastructure.Persistence.DatabaseModelDefinitions;
using Umbraco.Cms.Infrastructure.Persistence.Dtos;
using Umbraco.Cms.Persistence.SqlServer.Dtos;
using Umbraco.Extensions;
using ColumnInfo = Umbraco.Cms.Infrastructure.Persistence.SqlSyntax.ColumnInfo;
namespace Umbraco.Cms.Infrastructure.Persistence.SqlSyntax
namespace Umbraco.Cms.Persistence.SqlServer.Services
{
/// <summary>
/// Represents an SqlSyntaxProvider for Sql Server.
@@ -20,15 +19,22 @@ namespace Umbraco.Cms.Infrastructure.Persistence.SqlSyntax
public class SqlServerSyntaxProvider : MicrosoftSqlSyntaxProviderBase<SqlServerSyntaxProvider>
{
private readonly IOptions<GlobalSettings> _globalSettings;
private readonly ILogger<SqlServerSyntaxProvider> _logger;
public SqlServerSyntaxProvider(IOptions<GlobalSettings> globalSettings)
: this(globalSettings, StaticApplicationLogging.CreateLogger<SqlServerSyntaxProvider>())
{
_globalSettings = globalSettings;
}
public override string ProviderName => Cms.Core.Constants.DatabaseProviders.SqlServer;
public SqlServerSyntaxProvider(IOptions<GlobalSettings> globalSettings, ILogger<SqlServerSyntaxProvider> logger)
{
_globalSettings = globalSettings;
_logger = logger;
}
public ServerVersionInfo ServerVersion { get; private set; }
public override string ProviderName => Constants.ProviderName;
public ServerVersionInfo? ServerVersion { get; private set; }
public enum VersionName
{
@@ -56,6 +62,22 @@ namespace Umbraco.Cms.Infrastructure.Persistence.SqlSyntax
Azure = 5
}
public override DatabaseType GetUpdatedDatabaseType(DatabaseType current, string connectionString)
{
var setting = _globalSettings.Value.DatabaseFactoryServerVersion;
var fromSettings = false;
if (setting.IsNullOrWhiteSpace() || !setting.StartsWith("SqlServer.")
|| !Enum<SqlServerSyntaxProvider.VersionName>.TryParse(setting.Substring("SqlServer.".Length), out var versionName, true))
{
versionName = GetSetVersion(connectionString, ProviderName, _logger).ProductVersionName;
}
_logger.LogDebug("SqlServer {SqlServerVersion}, DatabaseType is {DatabaseType} ({Source}).", versionName, DatabaseType.SqlServer2012, fromSettings ? "settings" : "detected");
return DatabaseType.SqlServer2012;
}
public class ServerVersionInfo
{
public ServerVersionInfo()
@@ -87,7 +109,7 @@ namespace Umbraco.Cms.Infrastructure.Persistence.SqlSyntax
private static VersionName MapProductVersion(string productVersion)
{
var firstPart = string.IsNullOrWhiteSpace(productVersion) ? "??" : productVersion.Split(Constants.CharArrays.Period)[0];
var firstPart = string.IsNullOrWhiteSpace(productVersion) ? "??" : productVersion.Split(Cms.Core.Constants.CharArrays.Period)[0];
switch (firstPart)
{
case "??":
@@ -266,107 +288,6 @@ where tbl.[name]=@0 and col.[name]=@1;", tableName, columnName)
return result > 0;
}
public override void WriteLock(IDatabase db, TimeSpan timeout, int lockId)
{
// soon as we get Database, a transaction is started
if (db.Transaction.IsolationLevel < IsolationLevel.ReadCommitted)
throw new InvalidOperationException("A transaction with minimum ReadCommitted isolation level is required.");
ObtainWriteLock(db, timeout, lockId);
}
public override void WriteLock(IDatabase db, params int[] lockIds)
{
WriteLock(db, _globalSettings.Value.SqlWriteLockTimeOut, lockIds);
}
public void WriteLock(IDatabase db, TimeSpan timeout, params int[] lockIds)
{
if (db is null)
{
throw new ArgumentNullException(nameof(db));
}
if (db.Transaction is null)
{
throw new ArgumentException(nameof(db) + "." + nameof(db.Transaction) + " is null");
}
// soon as we get Database, a transaction is started
if (db.Transaction.IsolationLevel < IsolationLevel.ReadCommitted)
{
throw new InvalidOperationException("A transaction with minimum ReadCommitted isolation level is required.");
}
foreach (var lockId in lockIds)
{
ObtainWriteLock(db, timeout, lockId);
}
}
private static void ObtainWriteLock(IDatabase db, TimeSpan timeout, int lockId)
{
db.Execute("SET LOCK_TIMEOUT " + timeout.TotalMilliseconds + ";");
var i = db.Execute(
@"UPDATE umbracoLock WITH (REPEATABLEREAD) SET value = (CASE WHEN (value=1) THEN -1 ELSE 1 END) WHERE id=@id",
new {id = lockId});
if (i == 0) // ensure we are actually locking!
{
throw new ArgumentException($"LockObject with id={lockId} does not exist.");
}
}
public override void ReadLock(IDatabase db, TimeSpan timeout, int lockId)
{
// soon as we get Database, a transaction is started
if (db.Transaction.IsolationLevel < IsolationLevel.ReadCommitted)
throw new InvalidOperationException("A transaction with minimum RepeatableRead isolation level is required.");
ObtainReadLock(db, timeout, lockId);
}
public override void ReadLock(IDatabase db, params int[] lockIds)
{
if (db is null)
{
throw new ArgumentNullException(nameof(db));
}
if (db.Transaction is null)
{
throw new ArgumentException(nameof(db) + "." + nameof(db.Transaction) + " is null");
}
// soon as we get Database, a transaction is started
if (db.Transaction.IsolationLevel < IsolationLevel.ReadCommitted)
{
throw new InvalidOperationException("A transaction with minimum ReadCommitted isolation level is required.");
}
foreach (var lockId in lockIds)
{
ObtainReadLock(db, null, lockId);
}
}
private static void ObtainReadLock(IDatabase db, TimeSpan? timeout, int lockId)
{
if (timeout.HasValue)
{
db.Execute(@"SET LOCK_TIMEOUT " + timeout.Value.TotalMilliseconds + ";");
}
var i = db.ExecuteScalar<int?>("SELECT value FROM umbracoLock WITH (REPEATABLEREAD) WHERE id=@id", new {id = lockId});
if (i == null) // ensure we are actually locking!
{
throw new ArgumentException($"LockObject with id={lockId} does not exist.", nameof(lockId));
}
}
public override string FormatColumnRename(string tableName, string oldName, string newName)
{
return string.Format(RenameColumn, tableName, oldName, newName);
@@ -433,5 +354,90 @@ where tbl.[name]=@0 and col.[name]=@1;", tableName, columnName)
return string.Format(CreateIndex, GetIndexType(index.IndexType), " ", GetQuotedName(name),
GetQuotedTableName(index.TableName), columns, includeColumns);
}
public override Sql<ISqlContext> InsertForUpdateHint(Sql<ISqlContext> sql)
{
// go find the first FROM clause, and append the lock hint
Sql s = sql;
var updated = false;
while (s != null)
{
var sqlText = SqlInspector.GetSqlText(s);
if (sqlText.StartsWith("FROM ", StringComparison.OrdinalIgnoreCase))
{
SqlInspector.SetSqlText(s, sqlText + " WITH (UPDLOCK)");
updated = true;
break;
}
s = SqlInspector.GetSqlRhs(sql);
}
if (updated)
SqlInspector.Reset(sql);
return sql;
}
public override Sql<ISqlContext> AppendForUpdateHint(Sql<ISqlContext> sql)
=> sql.Append(" WITH (UPDLOCK) ");
public override Sql<ISqlContext>.SqlJoinClause<ISqlContext> LeftJoinWithNestedJoin<TDto>(
Sql<ISqlContext> sql,
Func<Sql<ISqlContext>,
Sql<ISqlContext>> nestedJoin,
string? alias = null)
{
Type type = typeof(TDto);
var tableName = GetQuotedTableName(type.GetTableName());
var join = tableName;
if (alias != null)
{
var quotedAlias = GetQuotedTableName(alias);
join += " " + quotedAlias;
}
var nestedSql = new Sql<ISqlContext>(sql.SqlContext);
nestedSql = nestedJoin(nestedSql);
Sql<ISqlContext>.SqlJoinClause<ISqlContext> sqlJoin = sql.LeftJoin(join);
sql.Append(nestedSql);
return sqlJoin;
}
#region Sql Inspection
private static SqlInspectionUtilities _sqlInspector;
private static SqlInspectionUtilities SqlInspector => _sqlInspector ?? (_sqlInspector = new SqlInspectionUtilities());
private class SqlInspectionUtilities
{
private readonly Func<Sql, string> _getSqlText;
private readonly Action<Sql, string> _setSqlText;
private readonly Func<Sql, Sql> _getSqlRhs;
private readonly Action<Sql, string> _setSqlFinal;
public SqlInspectionUtilities()
{
(_getSqlText, _setSqlText) = ReflectionUtilities.EmitFieldGetterAndSetter<Sql, string>("_sql");
_getSqlRhs = ReflectionUtilities.EmitFieldGetter<Sql, Sql>("_rhs");
_setSqlFinal = ReflectionUtilities.EmitFieldSetter<Sql, string>("_sqlFinal");
}
public string GetSqlText(Sql sql) => _getSqlText(sql);
public void SetSqlText(Sql sql, string sqlText) => _setSqlText(sql, sqlText);
public Sql GetSqlRhs(Sql sql) => _getSqlRhs(sql);
public void Reset(Sql sql) => _setSqlFinal(sql, null);
}
#endregion
}
}

14
src/Umbraco.Cms.Persistence.SqlServer/SqlServerComposer.cs Normal file
View File

@@ -0,0 +1,14 @@
using Umbraco.Cms.Core.Composing;
using Umbraco.Cms.Core.DependencyInjection;
namespace Umbraco.Cms.Persistence.SqlServer;
/// <summary>
/// Automatically adds SQL Server support to Umbraco when this project is referenced.
/// </summary>
public class SqlServerComposer : IComposer
{
/// <inheritdoc />
public void Compose(IUmbracoBuilder builder)
=> builder.AddUmbracoSqlServerSupport();
}

26
src/Umbraco.Cms.Persistence.SqlServer/Umbraco.Cms.Persistence.SqlServer.csproj Normal file
View File

@@ -0,0 +1,26 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>net6.0</TargetFramework>
<ImplicitUsings>enable</ImplicitUsings>
<Nullable>enable</Nullable>
<PackageId>Umbraco.Cms.Persistence.SqlServer</PackageId>
<Title>Umbraco.Cms.Persistence.SqlServer</Title>
<Description>Adds support for SQL Server to Umbraco CMS.</Description>
</PropertyGroup>
<ItemGroup>
<ProjectReference Include="..\Umbraco.Infrastructure\Umbraco.Infrastructure.csproj" />
</ItemGroup>
<ItemGroup>
<AssemblyAttribute Include="System.Runtime.CompilerServices.InternalsVisibleTo">
<_Parameter1>Umbraco.Tests.Integration</_Parameter1>
</AssemblyAttribute>
<AssemblyAttribute Include="System.Runtime.CompilerServices.InternalsVisibleTo">
<_Parameter1>Umbraco.Tests.UnitTests</_Parameter1>
</AssemblyAttribute>
</ItemGroup>
</Project>

42
src/Umbraco.Cms.Persistence.SqlServer/UmbracoBuilderExtensions.cs Normal file
View File

@@ -0,0 +1,42 @@
using System.Data.Common;
using Microsoft.Data.SqlClient;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.DependencyInjection.Extensions;
using Umbraco.Cms.Core.DependencyInjection;
using Umbraco.Cms.Core.DistributedLocking;
using Umbraco.Cms.Infrastructure.Persistence;
using Umbraco.Cms.Infrastructure.Persistence.SqlSyntax;
using Umbraco.Cms.Persistence.SqlServer.Interceptors;
using Umbraco.Cms.Persistence.SqlServer.Services;
namespace Umbraco.Cms.Persistence.SqlServer;
/// <summary>
/// SQLite support extensions for IUmbracoBuilder.
/// </summary>
public static class UmbracoBuilderExtensions
{
/// <summary>
/// Add required services for SQL Server support.
/// </summary>
public static IUmbracoBuilder AddUmbracoSqlServerSupport(this IUmbracoBuilder builder)
{
builder.Services.TryAddEnumerable(ServiceDescriptor.Singleton<ISqlSyntaxProvider, SqlServerSyntaxProvider>());
builder.Services.TryAddEnumerable(ServiceDescriptor.Singleton<IBulkSqlInsertProvider, SqlServerBulkSqlInsertProvider>());
builder.Services.TryAddEnumerable(ServiceDescriptor.Singleton<IDatabaseCreator, SqlServerDatabaseCreator>());
builder.Services.TryAddEnumerable(ServiceDescriptor.Singleton<IDatabaseProviderMetadata, SqlLocalDbDatabaseProviderMetadata>());
builder.Services.TryAddEnumerable(ServiceDescriptor.Singleton<IDatabaseProviderMetadata, SqlServerDatabaseProviderMetadata>());
builder.Services.TryAddEnumerable(ServiceDescriptor.Singleton<IDatabaseProviderMetadata, SqlAzureDatabaseProviderMetadata>());
builder.Services.TryAddEnumerable(ServiceDescriptor.Singleton<IDistributedLockingMechanism, SqlServerDistributedLockingMechanism>());
builder.Services.TryAddEnumerable(ServiceDescriptor.Singleton<IProviderSpecificInterceptor, SqlServerAddMiniProfilerInterceptor>());
builder.Services.TryAddEnumerable(ServiceDescriptor.Singleton<IProviderSpecificInterceptor, SqlServerAddRetryPolicyInterceptor>());
DbProviderFactories.UnregisterFactory(Constants.ProviderName);
DbProviderFactories.RegisterFactory(Constants.ProviderName, SqlClientFactory.Instance);
return builder;
}
}

12
src/Umbraco.Cms.Persistence.Sqlite/Constants.cs Normal file
View File

@@ -0,0 +1,12 @@
namespace Umbraco.Cms.Persistence.Sqlite;
/// <summary>
/// Constants related to SQLite.
/// </summary>
public static class Constants
{
/// <summary>
/// SQLite provider name.
/// </summary>
public const string ProviderName = "Microsoft.Data.SQLite";
}

11
src/Umbraco.Cms.Persistence.Sqlite/Interceptors/SqliteAddMiniProfilerInterceptor.cs Normal file
View File

@@ -0,0 +1,11 @@
using System.Data.Common;
using NPoco;
using StackExchange.Profiling;
namespace Umbraco.Cms.Persistence.Sqlite.Interceptors;
public class SqliteAddMiniProfilerInterceptor : SqliteConnectionInterceptor
{
public override DbConnection OnConnectionOpened(IDatabase database, DbConnection conn)
=> new StackExchange.Profiling.Data.ProfiledDbConnection(conn, MiniProfiler.Current);
}

12
src/Umbraco.Cms.Persistence.Sqlite/Interceptors/SqliteAddPreferDeferredInterceptor.cs Normal file
View File

@@ -0,0 +1,12 @@
using System.Data.Common;
using Microsoft.Data.Sqlite;
using NPoco;
using Umbraco.Cms.Persistence.Sqlite.Services;
namespace Umbraco.Cms.Persistence.Sqlite.Interceptors;
public class SqliteAddPreferDeferredInterceptor : SqliteConnectionInterceptor
{
public override DbConnection OnConnectionOpened(IDatabase database, DbConnection conn)
=> new SqlitePreferDeferredTransactionsConnection(conn as SqliteConnection ?? throw new InvalidOperationException());
}

16
src/Umbraco.Cms.Persistence.Sqlite/Interceptors/SqliteAddRetryPolicyInterceptor.cs Normal file
View File

@@ -0,0 +1,16 @@
using System.Data.Common;
using NPoco;
using Umbraco.Cms.Infrastructure.Persistence.FaultHandling;
using Umbraco.Cms.Persistence.Sqlite.Services;
namespace Umbraco.Cms.Persistence.Sqlite.Interceptors;
public class SqliteAddRetryPolicyInterceptor : SqliteConnectionInterceptor
{
public override DbConnection OnConnectionOpened(IDatabase database, DbConnection conn)
{
RetryStrategy retryStrategy = RetryStrategy.DefaultExponential;
var commandRetryPolicy = new RetryPolicy(new SqliteTransientErrorDetectionStrategy(), retryStrategy);
return new RetryDbConnection(conn, null, commandRetryPolicy);
}
}

16
src/Umbraco.Cms.Persistence.Sqlite/Interceptors/SqliteConnectionInterceptor.cs Normal file
View File

@@ -0,0 +1,16 @@
using System.Data.Common;
using NPoco;
using Umbraco.Cms.Infrastructure.Persistence;
namespace Umbraco.Cms.Persistence.Sqlite.Interceptors;
public abstract class SqliteConnectionInterceptor : IProviderSpecificConnectionInterceptor
{
public string ProviderName => Constants.ProviderName;
public abstract DbConnection OnConnectionOpened(IDatabase database, DbConnection conn);
public virtual void OnConnectionClosing(IDatabase database, DbConnection conn)
{
}
}

24
src/Umbraco.Cms.Persistence.Sqlite/Mappers/SqliteGuidScalarMapper.cs Normal file
View File

@@ -0,0 +1,24 @@
using Umbraco.Cms.Infrastructure.Persistence;
namespace Umbraco.Cms.Persistence.Sqlite.Mappers;
public class SqliteGuidScalarMapper : ScalarMapper<Guid>
{
protected override Guid Map(object value)
=> Guid.Parse($"{value}");
}
public class SqliteNullableGuidScalarMapper : ScalarMapper<Guid?>
{
protected override Guid? Map(object? value)
{
if (value is null || value == DBNull.Value)
{
return default(Guid?);
}
return Guid.TryParse($"{value}", out Guid result)
? result
: default(Guid?);
}
}

33
src/Umbraco.Cms.Persistence.Sqlite/Mappers/SqlitePocoGuidMapper.cs Normal file
View File

@@ -0,0 +1,33 @@
using NPoco;
namespace Umbraco.Cms.Persistence.Sqlite.Mappers;
public class SqlitePocoGuidMapper : DefaultMapper
{
public override Func<object, object?> GetFromDbConverter(Type destType, Type sourceType)
{
if (destType == typeof(Guid))
{
return (value) =>
{
var result = Guid.Parse($"{value}");
return result;
};
}
if (destType == typeof(Guid?))
{
return (value) =>
{
if (Guid.TryParse($"{value}", out Guid result))
{
return result;
}
return default(Guid?);
};
}
return base.GetFromDbConverter(destType, sourceType);
}
}

55
src/Umbraco.Cms.Persistence.Sqlite/Services/SqliteBulkSqlInsertProvider.cs Normal file
View File

@@ -0,0 +1,55 @@
using NPoco;
using Umbraco.Cms.Infrastructure.Persistence;
namespace Umbraco.Cms.Persistence.Sqlite.Services;
/// <summary>
/// Implements <see cref="IBulkSqlInsertProvider"/> for SQLite.
/// </summary>
public class SqliteBulkSqlInsertProvider : IBulkSqlInsertProvider
{
public string ProviderName => Constants.ProviderName;
public int BulkInsertRecords<T>(IUmbracoDatabase database, IEnumerable<T> records)
{
var recordsA = records.ToArray();
if (recordsA.Length == 0) return 0;
var pocoData = database.PocoDataFactory.ForType(typeof(T));
if (pocoData == null) throw new InvalidOperationException("Could not find PocoData for " + typeof(T));
return BulkInsertRecordsSqlite(database, pocoData, recordsA);
}
/// <summary>
/// Bulk-insert records using SqlServer BulkCopy method.
/// </summary>
/// <typeparam name="T">The type of the records.</typeparam>
/// <param name="database">The database.</param>
/// <param name="pocoData">The PocoData object corresponding to the record's type.</param>
/// <param name="records">The records.</param>
/// <returns>The number of records that were inserted.</returns>
private int BulkInsertRecordsSqlite<T>(IUmbracoDatabase database, PocoData pocoData, IEnumerable<T> records)
{
var count = 0;
var inTrans = database.InTransaction;
if (!inTrans)
{
database.BeginTransaction();
}
foreach (var record in records)
{
database.Insert(record);
count++;
}
if (!inTrans)
{
database.CompleteTransaction();
}
return count;
}
}

48
src/Umbraco.Cms.Persistence.Sqlite/Services/SqliteDatabaseCreator.cs Normal file
View File

@@ -0,0 +1,48 @@
using System.Diagnostics;
using Microsoft.Data.Sqlite;
using Umbraco.Cms.Infrastructure.Persistence;
namespace Umbraco.Cms.Persistence.Sqlite.Services;
/// <summary>
/// Implements <see cref="IDatabaseCreator"/> for SQLite.
/// </summary>
public class SqliteDatabaseCreator : IDatabaseCreator
{
/// <inheritdoc />
public string ProviderName => Constants.ProviderName;
/// <summary>
/// Creates a SQLite database file.
/// </summary>
/// <remarks>
/// <para>
/// With journal_mode = wal we have snapshot isolation.
/// </para>
/// <para>
/// Concurrent read/write can take occur, committing a write transaction will have no impact
/// on open read transactions as they see only committed data from the point in time that they began reading.
/// </para>
/// <para>
/// A write transaction still requires exclusive access to database files so concurrent writes are not possible.
/// </para>
/// <para>
/// Read more <a href="https://www.sqlite.org/isolation.html">Isolation in SQLite</a> <br/>
/// Read more <a href="https://www.sqlite.org/wal.html">Write-Ahead Logging</a>
/// </para>
/// </remarks>
public void Create(string connectionString)
{
using var connection = new SqliteConnection(connectionString);
connection.Open();
using SqliteCommand command = connection.CreateCommand();
command.CommandText = "PRAGMA journal_mode = wal;";
command.ExecuteNonQuery();
command.CommandText = "PRAGMA journal_mode";
var mode = command.ExecuteScalar();
Debug.Assert(mode as string == "wal", "incorrect journal_mode");
}
}

72
src/Umbraco.Cms.Persistence.Sqlite/Services/SqliteDatabaseProviderMetadata.cs Normal file
View File

@@ -0,0 +1,72 @@
using System.Runtime.Serialization;
using Microsoft.Data.Sqlite;
using Umbraco.Cms.Core.Configuration.Models;
using Umbraco.Cms.Core.Install.Models;
using Umbraco.Cms.Infrastructure.Persistence;
namespace Umbraco.Cms.Persistence.Sqlite.Services;
[DataContract]
public class SqliteDatabaseProviderMetadata : IDatabaseProviderMetadata
{
/// <inheritdoc />
public Guid Id => new ("530386a2-b219-4d5f-b68c-b965e14c9ac9");
/// <inheritdoc />
public int SortOrder => -1;
/// <inheritdoc />
public string DisplayName => "SQLite";
/// <inheritdoc />
public string DefaultDatabaseName => Core.Constants.System.UmbracoDefaultDatabaseName;
/// <inheritdoc />
public string ProviderName => Constants.ProviderName;
/// <inheritdoc />
public bool SupportsQuickInstall => true;
/// <inheritdoc />
public bool IsAvailable => true;
/// <inheritdoc />
public bool RequiresServer => false;
/// <inheritdoc />
public string ServerPlaceholder => null;
/// <inheritdoc />
public bool RequiresCredentials => false;
/// <inheritdoc />
public bool SupportsIntegratedAuthentication => false;
/// <inheritdoc />
public bool RequiresConnectionTest => false;
/// <inheritdoc />
/// <remarks>
/// <para>
/// Required to ensure database creator is used regardless of configured InstallMissingDatabase value.
/// </para>
/// <para>
/// Ensures database setup with journal_mode = wal;
/// </para>
/// </remarks>
public bool ForceCreateDatabase => true;
/// <inheritdoc />
public string GenerateConnectionString(DatabaseModel databaseModel)
{
var builder = new SqliteConnectionStringBuilder
{
DataSource = $"{ConnectionStrings.DataDirectoryPlaceholder}/{databaseModel.DatabaseName}.sqlite.db",
ForeignKeys = true,
Pooling = true,
Cache = SqliteCacheMode.Shared,
};
return builder.ConnectionString;
}
}

157
src/Umbraco.Cms.Persistence.Sqlite/Services/SqliteDistributedLockingMechanism.cs Normal file
View File

@@ -0,0 +1,157 @@
using System.Data;
using System.Data.Common;
using Microsoft.Data.SqlClient;
using Microsoft.Data.Sqlite;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Options;
using Umbraco.Cms.Core.Configuration.Models;
using Umbraco.Cms.Core.DistributedLocking;
using Umbraco.Cms.Core.DistributedLocking.Exceptions;
using Umbraco.Cms.Infrastructure.Persistence;
using Umbraco.Cms.Infrastructure.Scoping;
using Umbraco.Extensions;
namespace Umbraco.Cms.Persistence.Sqlite.Services;
public class SqliteDistributedLockingMechanism : IDistributedLockingMechanism
{
private readonly ILogger<SqliteDistributedLockingMechanism> _logger;
private readonly Lazy<IScopeAccessor> _scopeAccessor;
private readonly IOptionsMonitor<ConnectionStrings> _connectionStrings;
private readonly IOptionsMonitor<GlobalSettings> _globalSettings;
public SqliteDistributedLockingMechanism(
ILogger<SqliteDistributedLockingMechanism> logger,
Lazy<IScopeAccessor> scopeAccessor,
IOptionsMonitor<GlobalSettings> globalSettings,
IOptionsMonitor<ConnectionStrings> connectionStrings)
{
_logger = logger;
_scopeAccessor = scopeAccessor;
_connectionStrings = connectionStrings;
_globalSettings = globalSettings;
}
/// <inheritdoc />
public bool Enabled => _connectionStrings.CurrentValue.IsConnectionStringConfigured() &&
_connectionStrings.CurrentValue.ProviderName == Constants.ProviderName;
// With journal_mode=wal we can always read a snapshot.
public IDistributedLock ReadLock(int lockId, TimeSpan? obtainLockTimeout = null)
{
obtainLockTimeout ??= _globalSettings.CurrentValue.DistributedLockingReadLockDefaultTimeout;
return new SqliteDistributedLock(this, lockId, DistributedLockType.ReadLock, obtainLockTimeout.Value);
}
// With journal_mode=wal only a single write transaction can exist at a time.
public IDistributedLock WriteLock(int lockId, TimeSpan? obtainLockTimeout = null)
{
obtainLockTimeout ??= _globalSettings.CurrentValue.DistributedLockingWriteLockDefaultTimeout;
return new SqliteDistributedLock(this, lockId, DistributedLockType.WriteLock, obtainLockTimeout.Value);
}
private class SqliteDistributedLock : IDistributedLock
{
private readonly SqliteDistributedLockingMechanism _parent;
private readonly TimeSpan _timeout;
public SqliteDistributedLock(
SqliteDistributedLockingMechanism parent,
int lockId,
DistributedLockType lockType,
TimeSpan timeout)
{
_parent = parent;
_timeout = timeout;
LockId = lockId;
LockType = lockType;
_parent._logger.LogDebug("Requesting {lockType} for id {id}", LockType, LockId);
try
{
switch (lockType)
{
case DistributedLockType.ReadLock:
ObtainReadLock();
break;
case DistributedLockType.WriteLock:
ObtainWriteLock();
break;
default:
throw new ArgumentOutOfRangeException(nameof(lockType), lockType, @"Unsupported lockType");
}
}
catch (SqlException ex) when (ex.Number == 1222)
{
if (LockType == DistributedLockType.ReadLock)
{
throw new DistributedReadLockTimeoutException(LockId);
}
throw new DistributedWriteLockTimeoutException(LockId);
}
_parent._logger.LogDebug("Acquired {lockType} for id {id}", LockType, LockId);
}
public int LockId { get; }
public DistributedLockType LockType { get; }
public void Dispose()
{
// Mostly no op, cleaned up by completing transaction in scope.
_parent._logger.LogDebug("Dropped {lockType} for id {id}", LockType, LockId);
}
public override string ToString()
=> $"SqliteDistributedLock({LockId})";
// Can always obtain a read lock (snapshot isolation in wal mode)
// Mostly no-op just check that we didn't end up ReadUncommitted for real.
private void ObtainReadLock()
{
IUmbracoDatabase db = _parent._scopeAccessor.Value.AmbientScope.Database;
if (!db.InTransaction)
{
throw new InvalidOperationException("SqliteDistributedLockingMechanism requires a transaction to function.");
}
}
// Only one writer is possible at a time
// lock occurs for entire database as opposed to row/table.
private void ObtainWriteLock()
{
IUmbracoDatabase db = _parent._scopeAccessor.Value.AmbientScope.Database;
if (!db.InTransaction)
{
throw new InvalidOperationException("SqliteDistributedLockingMechanism requires a transaction to function.");
}
var query = @$"UPDATE umbracoLock SET value = (CASE WHEN (value=1) THEN -1 ELSE 1 END) WHERE id = {LockId}";
DbCommand command = db.CreateCommand(db.Connection, CommandType.Text, query);
// imagine there is an existing writer, whilst elapsed time is < command timeout sqlite will busy loop
command.CommandTimeout = _timeout.Seconds;
try
{
var i = command.ExecuteNonQuery();
if (i == 0)
{
// ensure we are actually locking!
throw new ArgumentException($"LockObject with id={LockId} does not exist.");
}
}
catch (SqliteException ex) when (ex.IsBusyOrLocked())
{
throw new DistributedWriteLockTimeoutException(LockId);
}
}
}
}

12
src/Umbraco.Cms.Persistence.Sqlite/Services/SqliteExceptionExtensions.cs Normal file
View File

@@ -0,0 +1,12 @@
using Microsoft.Data.Sqlite;
namespace Umbraco.Cms.Persistence.Sqlite.Services;
public static class SqliteExceptionExtensions
{
public static bool IsBusyOrLocked(this SqliteException ex) =>
ex.SqliteErrorCode
is SQLitePCL.raw.SQLITE_BUSY
or SQLitePCL.raw.SQLITE_LOCKED
or SQLitePCL.raw.SQLITE_LOCKED_SHAREDCACHE;
}

124
src/Umbraco.Cms.Persistence.Sqlite/Services/SqlitePreferDeferredTransactionsConnection.cs Normal file
View File

@@ -0,0 +1,124 @@
using System.Data;
using System.Data.Common;
using Microsoft.Data.Sqlite;
namespace Umbraco.Cms.Persistence.Sqlite.Services;
public class SqlitePreferDeferredTransactionsConnection : DbConnection
{
private readonly SqliteConnection _inner;
public SqlitePreferDeferredTransactionsConnection(SqliteConnection inner)
{
_inner = inner;
}
protected override DbTransaction BeginDbTransaction(IsolationLevel isolationLevel)
=> _inner.BeginTransaction(isolationLevel, deferred: true); // <-- The important bit
public override void ChangeDatabase(string databaseName)
=> _inner.ChangeDatabase(databaseName);
public override void Close()
=> _inner.Close();
public override void Open()
=> _inner.Open();
public override string Database
=> _inner.Database;
public override ConnectionState State
=> _inner.State;
public override string DataSource
=> _inner.DataSource;
public override string ServerVersion
=> _inner.ServerVersion;
protected override DbCommand CreateDbCommand()
=> new CommandWrapper(_inner.CreateCommand());
public override string ConnectionString
{
get => _inner.ConnectionString;
set => _inner.ConnectionString = value;
}
private class CommandWrapper : DbCommand
{
private readonly DbCommand _inner;
public CommandWrapper(DbCommand inner)
{
_inner = inner;
}
public override void Cancel()
=> _inner.Cancel();
public override int ExecuteNonQuery()
=> _inner.ExecuteNonQuery();
public override object? ExecuteScalar()
=> _inner.ExecuteScalar();
public override void Prepare()
=> _inner.Prepare();
public override string CommandText
{
get => _inner.CommandText;
set => _inner.CommandText = value;
}
public override int CommandTimeout
{
get => _inner.CommandTimeout;
set => _inner.CommandTimeout = value;
}
public override CommandType CommandType
{
get => _inner.CommandType;
set => _inner.CommandType = value;
}
public override UpdateRowSource UpdatedRowSource
{
get => _inner.UpdatedRowSource;
set => _inner.UpdatedRowSource = value;
}
protected override DbConnection? DbConnection
{
get => _inner.Connection;
set
{
_inner.Connection = (value as SqlitePreferDeferredTransactionsConnection)?._inner;
}
}
protected override DbParameterCollection DbParameterCollection
=> _inner.Parameters;
protected override DbTransaction? DbTransaction
{
get => _inner.Transaction;
set => _inner.Transaction = value;
}
public override bool DesignTimeVisible
{
get => _inner.DesignTimeVisible;
set => _inner.DesignTimeVisible = value;
}
protected override DbParameter CreateDbParameter()
=> _inner.CreateParameter();
protected override DbDataReader ExecuteDbDataReader(CommandBehavior behavior)
=> _inner.ExecuteReader(behavior);
}
}

16
src/Umbraco.Cms.Persistence.Sqlite/Services/SqliteSpecificMapperFactory.cs Normal file
View File

@@ -0,0 +1,16 @@
using Umbraco.Cms.Infrastructure.Persistence;
using Umbraco.Cms.Persistence.Sqlite.Mappers;
namespace Umbraco.Cms.Persistence.Sqlite.Services;
/// <summary>
/// Implements <see cref="IProviderSpecificMapperFactory"/> for SQLite.
/// </summary>
public class SqliteSpecificMapperFactory : IProviderSpecificMapperFactory
{
/// <inheritdoc />
public string ProviderName => Constants.ProviderName;
/// <inheritdoc />
public NPocoMapperCollection Mappers => new NPocoMapperCollection(() => new[] { new SqlitePocoGuidMapper() });
}

424
src/Umbraco.Cms.Persistence.Sqlite/Services/SqliteSyntaxProvider.cs Normal file
View File

@@ -0,0 +1,424 @@
using System.Data;
using System.Data.Common;
using System.Linq.Expressions;
using System.Reflection;
using System.Text;
using System.Text.RegularExpressions;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Options;
using NPoco;
using Umbraco.Cms.Core;
using Umbraco.Cms.Core.Configuration.Models;
using Umbraco.Cms.Infrastructure.Persistence;
using Umbraco.Cms.Infrastructure.Persistence.DatabaseAnnotations;
using Umbraco.Cms.Infrastructure.Persistence.DatabaseModelDefinitions;
using Umbraco.Cms.Infrastructure.Persistence.SqlSyntax;
using Umbraco.Cms.Persistence.Sqlite.Mappers;
using Umbraco.Extensions;
using ColumnInfo = Umbraco.Cms.Infrastructure.Persistence.SqlSyntax.ColumnInfo;
namespace Umbraco.Cms.Persistence.Sqlite.Services;
/// <summary>
/// Implements <see cref="ISqlSyntaxProvider"/> for SQLite.
/// </summary>
public class SqliteSyntaxProvider : SqlSyntaxProviderBase<SqliteSyntaxProvider>
{
private readonly IOptions<GlobalSettings> _globalSettings;
private readonly ILogger<SqliteSyntaxProvider> _log;
private readonly IDictionary<Type, IScalarMapper> _scalarMappers;
public SqliteSyntaxProvider(IOptions<GlobalSettings> globalSettings, ILogger<SqliteSyntaxProvider> log)
{
_globalSettings = globalSettings;
_log = log;
_scalarMappers = new Dictionary<Type, IScalarMapper>
{
[typeof(Guid)] = new SqliteGuidScalarMapper(),
[typeof(Guid?)] = new SqliteNullableGuidScalarMapper(),
};
}
/// <inheritdoc />
public override string ProviderName => Constants.ProviderName;
public override string StringColumnDefinition => "TEXT COLLATE NOCASE";
public override string StringLengthUnicodeColumnDefinitionFormat => "TEXT COLLATE NOCASE";
/// <inheritdoc />
public override IsolationLevel DefaultIsolationLevel
=> IsolationLevel.Serializable;
/// <inheritdoc />
public override string DbProvider => Constants.ProviderName;
/// <inheritdoc />
public override bool SupportsIdentityInsert() => false;
/// <inheritdoc />
public override bool SupportsClustered() => false;
public override string GetIndexType(IndexTypes indexTypes)
{
switch (indexTypes)
{
case IndexTypes.UniqueNonClustered:
return "UNIQUE";
default:
return string.Empty;
}
}
public override List<string> Format(IEnumerable<ForeignKeyDefinition> foreignKeys)
{
return foreignKeys.Select(Format).ToList();
}
public virtual string Format(ForeignKeyDefinition foreignKey)
{
var constraintName = string.IsNullOrEmpty(foreignKey.Name)
? $"FK_{foreignKey.ForeignTable}_{foreignKey.PrimaryTable}_{foreignKey.PrimaryColumns.First()}"
: foreignKey.Name;
var localColumn = GetQuotedColumnName(foreignKey.ForeignColumns.First());
var remoteColumn = GetQuotedColumnName(foreignKey.PrimaryColumns.First());
var remoteTable = GetQuotedTableName(foreignKey.PrimaryTable);
var onDelete = FormatCascade("DELETE", foreignKey.OnDelete);
var onUpdate = FormatCascade("UPDATE", foreignKey.OnUpdate);
return
$"CONSTRAINT {constraintName} FOREIGN KEY ({localColumn}) REFERENCES {remoteTable} ({remoteColumn}) {onDelete} {onUpdate}";
}
/// <inheritdoc />
public override IEnumerable<Tuple<string, string, string, bool>> GetDefinedIndexes(IDatabase db)
{
List<IndexMeta> items = db.Fetch<IndexMeta>(
@"SELECT
m.tbl_name AS tableName,
ilist.name AS indexName,
iinfo.name AS columnName,
ilist.[unique] AS isUnique
FROM
sqlite_master AS m,
pragma_index_list(m.name) AS ilist,
pragma_index_info(ilist.name) AS iinfo");
return items
.Where(x => !x.IndexName.StartsWith("sqlite_"))
.Select(item =>
new Tuple<string, string, string, bool>(item.TableName, item.IndexName, item.ColumnName, item.IsUnique))
.ToList();
}
public override string ConvertIntegerToOrderableString => "substr('0000000000'||'{0}', -10, 10)";
public override string ConvertDecimalToOrderableString => "substr('0000000000'||'{0}', -10, 10)";
public override string ConvertDateToOrderableString => "{0}";
/// <inheritdoc />
public override string GetSpecialDbType(SpecialDbType dbType) => "TEXT COLLATE NOCASE";
/// <inheritdoc />
public override string GetSpecialDbType(SpecialDbType dbType, int customSize) => GetSpecialDbType(dbType);
/// <inheritdoc />
public override bool TryGetDefaultConstraint(IDatabase db, string tableName, string columnName,
out string constraintName)
{
// TODO: SQLite
constraintName = string.Empty;
return false;
}
public override string GetFieldNameForUpdate<TDto>(Expression<Func<TDto, object>> fieldSelector,
string tableAlias = null)
{
var field = ExpressionHelper.FindProperty(fieldSelector).Item1 as PropertyInfo;
var fieldName = GetColumnName(field!);
return GetQuotedColumnName(fieldName);
}
private static string GetColumnName(PropertyInfo column)
{
ColumnAttribute? attr = column.FirstAttribute<ColumnAttribute>();
return string.IsNullOrWhiteSpace(attr?.Name) ? column.Name : attr.Name;
}
/// <inheritdoc />
protected override string FormatSystemMethods(SystemMethods systemMethod)
{
// TODO: SQLite
switch (systemMethod)
{
case SystemMethods.NewGuid:
return "NEWID()"; // No NEWID() in SQLite perhaps try RANDOM()
case SystemMethods.CurrentDateTime:
return "DATE()"; // No GETDATE() trying DATE()
}
return null;
}
/// <inheritdoc />
protected override string FormatIdentity(ColumnDefinition column)
{
/* NOTE: We need AUTOINCREMENT, adds overhead but makes magic ids not break everything.
* e.g. Cms.Core.Constants.Security.SuperUserId is -1
* without the sqlite_sequence table we end up with the next user id = 0
* but 0 is considered to not exist by our c# code and things explode */
return column.IsIdentity ? "PRIMARY KEY AUTOINCREMENT" : string.Empty;
}
public override string GetConcat(params string[] args)
{
return string.Join(" || ", args.AsEnumerable());
}
public override string GetColumn(DatabaseType dbType, string tableName, string columnName, string columnAlias,
string referenceName = null, bool forInsert = false)
{
if (forInsert)
{
return dbType.EscapeSqlIdentifier(columnName);
}
return base.GetColumn(dbType, tableName, columnName, columnAlias, referenceName, forInsert);
}
public override string FormatPrimaryKey(TableDefinition table)
{
ColumnDefinition? columnDefinition = table.Columns.FirstOrDefault(x => x.IsPrimaryKey);
if (columnDefinition == null)
{
return string.Empty;
}
var constraintName = string.IsNullOrEmpty(columnDefinition.PrimaryKeyName)
? $"PK_{table.Name}"
: columnDefinition.PrimaryKeyName;
var columns = string.IsNullOrEmpty(columnDefinition.PrimaryKeyColumns)
? GetQuotedColumnName(columnDefinition.Name)
: string.Join(", ", columnDefinition.PrimaryKeyColumns
.Split(Cms.Core.Constants.CharArrays.CommaSpace, StringSplitOptions.RemoveEmptyEntries)
.Select(GetQuotedColumnName));
// We can't name the PK if it's set as a column constraint so add an alternate at table level.
var constraintType = table.Columns.Any(x => x.IsIdentity)
? "UNIQUE"
: "PRIMARY KEY";
return $"CONSTRAINT {constraintName} {constraintType} ({columns})";
}
/// <inheritdoc />
public override Sql<ISqlContext> SelectTop(Sql<ISqlContext> sql, int top)
{
// SQLite uses LIMIT as opposed to TOP
// SELECT TOP 5 * FROM My_Table
// SELECT * FROM My_Table LIMIT 5;
return sql.Append($"LIMIT {top}");
}
public virtual string Format(IEnumerable<ColumnDefinition> columns)
{
var sb = new StringBuilder();
foreach (ColumnDefinition column in columns)
{
sb.AppendLine(", " + Format(column));
}
return sb.ToString().TrimStart(',');
}
public override void HandleCreateTable(IDatabase database, TableDefinition tableDefinition,
bool skipKeysAndIndexes = false)
{
var columns = Format(tableDefinition.Columns);
var primaryKey = FormatPrimaryKey(tableDefinition);
List<string> foreignKeys = Format(tableDefinition.ForeignKeys);
var sb = new StringBuilder();
sb.AppendLine($"CREATE TABLE {tableDefinition.Name}");
sb.AppendLine("(");
sb.Append(columns);
if (!string.IsNullOrEmpty(primaryKey) && !skipKeysAndIndexes)
{
sb.AppendLine($", {primaryKey}");
}
if (!skipKeysAndIndexes)
{
foreach (var foreignKey in foreignKeys)
{
sb.AppendLine($", {foreignKey}");
}
}
sb.AppendLine(")");
var createSql = sb.ToString();
_log.LogInformation("Create table:\n {Sql}", createSql);
database.Execute(new Sql(createSql));
if (skipKeysAndIndexes)
{
return;
}
List<string> indexSql = Format(tableDefinition.Indexes);
foreach (var sql in indexSql)
{
_log.LogInformation("Create Index:\n {Sql}", sql);
database.Execute(new Sql(sql));
}
}
public override IEnumerable<string> GetTablesInSchema(IDatabase db) =>
db.Fetch<string>("select name from sqlite_master where type='table'")
.Where(x => !x.StartsWith("sqlite_"));
public override IEnumerable<ColumnInfo> GetColumnsInSchema(IDatabase db)
{
IEnumerable<string> tables = GetTablesInSchema(db);
db.OpenSharedConnection();
foreach (var table in tables)
{
DbCommand? cmd = db.CreateCommand(db.Connection, CommandType.Text, $"PRAGMA table_info({table})");
DbDataReader reader = cmd.ExecuteReader();
while (reader.Read())
{
var ordinal = reader.GetInt32("cid");
var columnName = reader.GetString("name");
var type = reader.GetString("type");
var notNull = reader.GetBoolean("notnull");
yield return new ColumnInfo(table, columnName, ordinal, notNull, type);
}
}
}
/// <inheritdoc />
public override IEnumerable<Tuple<string, string, string>> GetConstraintsPerColumn(IDatabase db)
{
var items = db.Fetch<SqliteMaster>("select * from sqlite_master where type = 'table'")
.Where(x => !x.Name.StartsWith("sqlite_"));
List<Constraint> foundConstraints = new();
foreach (SqliteMaster row in items)
{
var altPk = Regex.Match(row.Sql, @"CONSTRAINT (?<constraint>PK_\w+)\s.*UNIQUE \(""(?<field>.+?)""\)");
if (altPk.Success)
{
var field = altPk.Groups["field"].Value;
var constraint = altPk.Groups["constraint"].Value;
foundConstraints.Add(new Constraint(row.Name, field, constraint));
}
else
{
var identity = Regex.Match(row.Sql, @"""(?<field>.+)"".*AUTOINCREMENT");
if (identity.Success)
{
foundConstraints.Add(new Constraint(row.Name, identity.Groups["field"].Value, $"PK_{row.Name}"));
}
}
var pk = Regex.Match(row.Sql, @"CONSTRAINT (?<constraint>\w+)\s.*PRIMARY KEY \(""(?<field>.+?)""\)");
if (pk.Success)
{
var field = pk.Groups["field"].Value;
var constraint = pk.Groups["constraint"].Value;
foundConstraints.Add(new Constraint(row.Name, field, constraint));
}
var fkRegex = new Regex(@"CONSTRAINT (?<constraint>\w+) FOREIGN KEY \(""(?<field>.+?)""\) REFERENCES");
var foreignKeys = fkRegex.Matches(row.Sql).Cast<Match>();
{
foreach (var fk in foreignKeys)
{
var field = fk.Groups["field"].Value;
var constraint = fk.Groups["constraint"].Value;
foundConstraints.Add(new Constraint(row.Name, field, constraint));
}
}
}
// item.TableName, item.ColumnName, item.ConstraintName
return foundConstraints
.Select(x => Tuple.Create(x.TableName, x.ColumnName, x.ConstraintName));
}
public override Sql<ISqlContext>.SqlJoinClause<ISqlContext> LeftJoinWithNestedJoin<TDto>(
Sql<ISqlContext> sql,
Func<Sql<ISqlContext>,
Sql<ISqlContext>> nestedJoin,
string? alias = null)
{
Type type = typeof(TDto);
var tableName = GetQuotedTableName(type.GetTableName());
var join = tableName;
string? quotedAlias = null;
if (alias != null)
{
quotedAlias = GetQuotedTableName(alias);
join += " " + quotedAlias;
}
var nestedSql = new Sql<ISqlContext>(sql.SqlContext);
nestedSql = nestedJoin(nestedSql);
Sql<ISqlContext>.SqlJoinClause<ISqlContext> sqlJoin = sql.LeftJoin("(" + join);
sql.Append(nestedSql);
sql.Append($") {quotedAlias ?? tableName}");
return sqlJoin;
}
public override IDictionary<Type, IScalarMapper> ScalarMappers => _scalarMappers;
private class Constraint
{
public string TableName { get; }
public string ColumnName { get; }
public string ConstraintName { get; }
public Constraint(string tableName, string columnName, string constraintName)
{
TableName = tableName;
ColumnName = columnName;
ConstraintName = constraintName;
}
public override string ToString() => ConstraintName;
}
private class SqliteMaster
{
public string Type { get; set; }
public string Name { get; set; }
public string Sql { get; set; }
}
private class IndexMeta
{
public string TableName { get; set; }
public string IndexName { get; set; }
public string ColumnName { get; set; }
public bool IsUnique { get; set; }
}
}

17
src/Umbraco.Cms.Persistence.Sqlite/Services/SqliteTransientErrorDetectionStrategy.cs Normal file
View File

@@ -0,0 +1,17 @@
using Microsoft.Data.Sqlite;
using Umbraco.Cms.Infrastructure.Persistence.FaultHandling;
namespace Umbraco.Cms.Persistence.Sqlite.Services;
public class SqliteTransientErrorDetectionStrategy : ITransientErrorDetectionStrategy
{
public bool IsTransient(Exception ex)
{
if (ex is not SqliteException sqliteException)
{
return false;
}
return sqliteException.IsTransient || sqliteException.IsBusyOrLocked();
}
}

14
src/Umbraco.Cms.Persistence.Sqlite/SqliteComposer.cs Normal file
View File

@@ -0,0 +1,14 @@
using Umbraco.Cms.Core.Composing;
using Umbraco.Cms.Core.DependencyInjection;
namespace Umbraco.Cms.Persistence.Sqlite;
/// <summary>
/// Automatically adds SQLite support to Umbraco when this project is referenced.
/// </summary>
public class SqliteComposer : IComposer
{
/// <inheritdoc />
public void Compose(IUmbracoBuilder builder)
=> builder.AddUmbracoSqliteSupport();
}

21
src/Umbraco.Cms.Persistence.Sqlite/Umbraco.Cms.Persistence.Sqlite.csproj Normal file
View File

@@ -0,0 +1,21 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>net6.0</TargetFramework>
<ImplicitUsings>enable</ImplicitUsings>
<Nullable>enable</Nullable>
<PackageId>Umbraco.Cms.Persistence.Sqlite</PackageId>
<Title>Umbraco.Cms.Persistence.Sqlite</Title>
<Description>Adds support for SQLite to Umbraco CMS.</Description>
</PropertyGroup>
<ItemGroup>
<ProjectReference Include="..\Umbraco.Infrastructure\Umbraco.Infrastructure.csproj" />
</ItemGroup>
<ItemGroup>
<PackageReference Include="Microsoft.Data.Sqlite" Version="6.0.1" />
</ItemGroup>
</Project>

41
src/Umbraco.Cms.Persistence.Sqlite/UmbracoBuilderExtensions.cs Normal file
View File

@@ -0,0 +1,41 @@
using System.Data.Common;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.DependencyInjection.Extensions;
using Umbraco.Cms.Core.DependencyInjection;
using Umbraco.Cms.Core.DistributedLocking;
using Umbraco.Cms.Infrastructure.Persistence;
using Umbraco.Cms.Infrastructure.Persistence.SqlSyntax;
using Umbraco.Cms.Persistence.Sqlite.Interceptors;
using Umbraco.Cms.Persistence.Sqlite.Services;
namespace Umbraco.Cms.Persistence.Sqlite;
/// <summary>
/// SQLite support extensions for IUmbracoBuilder.
/// </summary>
public static class UmbracoBuilderExtensions
{
/// <summary>
/// Add required services for SQLite support.
/// </summary>
public static IUmbracoBuilder AddUmbracoSqliteSupport(this IUmbracoBuilder builder)
{
// TryAddEnumerable takes both TService and TImplementation into consideration (unlike TryAddSingleton)
builder.Services.TryAddEnumerable(ServiceDescriptor.Singleton<ISqlSyntaxProvider, SqliteSyntaxProvider>());
builder.Services.TryAddEnumerable(ServiceDescriptor.Singleton<IBulkSqlInsertProvider, SqliteBulkSqlInsertProvider>());
builder.Services.TryAddEnumerable(ServiceDescriptor.Singleton<IDatabaseCreator, SqliteDatabaseCreator>());
builder.Services.TryAddEnumerable(ServiceDescriptor.Singleton<IProviderSpecificMapperFactory, SqliteSpecificMapperFactory>());
builder.Services.TryAddEnumerable(ServiceDescriptor.Singleton<IDatabaseProviderMetadata, SqliteDatabaseProviderMetadata>());
builder.Services.TryAddEnumerable(ServiceDescriptor.Singleton<IDistributedLockingMechanism, SqliteDistributedLockingMechanism>());
builder.Services.TryAddEnumerable(ServiceDescriptor.Singleton<IProviderSpecificInterceptor, SqliteAddPreferDeferredInterceptor>());
builder.Services.TryAddEnumerable(ServiceDescriptor.Singleton<IProviderSpecificInterceptor, SqliteAddMiniProfilerInterceptor>());
builder.Services.TryAddEnumerable(ServiceDescriptor.Singleton<IProviderSpecificInterceptor, SqliteAddRetryPolicyInterceptor>());
DbProviderFactories.UnregisterFactory(Constants.ProviderName);
DbProviderFactories.RegisterFactory(Constants.ProviderName, Microsoft.Data.Sqlite.SqliteFactory.Instance);
return builder;
}
}

92
src/Umbraco.Core/Configuration/ConfigConnectionString.cs
View File

@@ -1,92 +0,0 @@
using System;
using System.Data.Common;
namespace Umbraco.Cms.Core.Configuration
{
public class ConfigConnectionString
{
public string Name { get; }
public string ConnectionString { get; }
public string ProviderName { get; }
public ConfigConnectionString(string name, string connectionString, string providerName = null)
{
Name = name ?? throw new ArgumentNullException(nameof(name));
ConnectionString = ParseConnectionString(connectionString, ref providerName);
ProviderName = providerName;
}
private static string ParseConnectionString(string connectionString, ref string providerName)
{
if (string.IsNullOrEmpty(connectionString))
{
return connectionString;
}
var builder = new DbConnectionStringBuilder
{
ConnectionString = connectionString
};
// Replace data directory placeholder
const string attachDbFileNameKey = "AttachDbFileName";
const string dataDirectoryPlaceholder = "|DataDirectory|";
if (builder.TryGetValue(attachDbFileNameKey, out var attachDbFileNameValue) &&
attachDbFileNameValue is string attachDbFileName &&
attachDbFileName.Contains(dataDirectoryPlaceholder))
{
var dataDirectory = AppDomain.CurrentDomain.GetData("DataDirectory")?.ToString();
if (!string.IsNullOrEmpty(dataDirectory))
{
builder[attachDbFileNameKey] = attachDbFileName.Replace(dataDirectoryPlaceholder, dataDirectory);
// Mutate the existing connection string (note: the builder also lowercases the properties)
connectionString = builder.ToString();
}
}
// Also parse provider name now we already have a builder
if (string.IsNullOrEmpty(providerName))
{
providerName = ParseProviderName(builder);
}
return connectionString;
}
/// <summary>
/// Parses the connection string to get the provider name.
/// </summary>
/// <param name="connectionString">The connection string.</param>
/// <returns>
/// The provider name or <c>null</c> is the connection string is empty.
/// </returns>
public static string ParseProviderName(string connectionString)
{
if (string.IsNullOrEmpty(connectionString))
{
return null;
}
var builder = new DbConnectionStringBuilder
{
ConnectionString = connectionString
};
return ParseProviderName(builder);
}
private static string ParseProviderName(DbConnectionStringBuilder builder)
{
if ((builder.TryGetValue("Data Source", out var dataSource) || builder.TryGetValue("DataSource", out dataSource)) &&
dataSource?.ToString().EndsWith(".sdf", StringComparison.OrdinalIgnoreCase) == true)
{
return Constants.DbProviderNames.SqlCe;
}
return Constants.DbProviderNames.SqlServer;
}
}
}

40
src/Umbraco.Core/Configuration/ConfigureConnectionStrings.cs Normal file
View File

@@ -0,0 +1,40 @@
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.Options;
using Umbraco.Cms.Core.Configuration.Models;
using Umbraco.Extensions;
namespace Umbraco.Cms.Core.Configuration;
/// <summary>
/// Configures ConnectionStrings.
/// </summary>
public class ConfigureConnectionStrings : IConfigureNamedOptions<ConnectionStrings>
{
private readonly IConfiguration _configuration;
/// <summary>
/// Initializes a new instance of the <see cref="ConfigureConnectionStrings"/> class.
/// </summary>
public ConfigureConnectionStrings(IConfiguration configuration) => _configuration = configuration;
/// <inheritdoc />
public void Configure(ConnectionStrings options) => Configure(Constants.System.UmbracoConnectionName, options);
/// <inheritdoc />
public void Configure(string name, ConnectionStrings options)
{
if (name == Options.DefaultName)
{
name = Constants.System.UmbracoConnectionName;
}
if (options.IsConnectionStringConfigured())
{
return;
}
options.Name = name;
options.ConnectionString = _configuration.GetConnectionString(name);
options.ProviderName = _configuration.GetConnectionString($"{name}{ConnectionStrings.ProviderNamePostfix}") ?? ConnectionStrings.DefaultProviderName;
}
}

55
src/Umbraco.Core/Configuration/Models/ConnectionStrings.cs
View File

@@ -1,31 +1,34 @@
// Copyright (c) Umbraco.
// See LICENSE for more details.
using Umbraco.Extensions;
namespace Umbraco.Cms.Core.Configuration.Models
namespace Umbraco.Cms.Core.Configuration.Models;
[UmbracoOptions("ConnectionStrings")]
public class ConnectionStrings
{
/// <summary>
/// Typed configuration options for connection strings.
/// </summary>
[UmbracoOptions("ConnectionStrings", BindNonPublicProperties = true)]
public class ConnectionStrings
{
// Backing field for UmbracoConnectionString to load from configuration value with key umbracoDbDSN.
// Attributes cannot be applied to map from keys that don't match, and have chosen to retain the key name
// used in configuration for older Umbraco versions.
// See: https://stackoverflow.com/a/54607296/489433
#pragma warning disable SA1300 // Element should begin with upper-case letter
#pragma warning disable IDE1006 // Naming Styles
private string umbracoDbDSN
#pragma warning restore IDE1006 // Naming Styles
#pragma warning restore SA1300 // Element should begin with upper-case letter
{
get => UmbracoConnectionString?.ConnectionString;
set => UmbracoConnectionString = new ConfigConnectionString(Constants.System.UmbracoConnectionName, value);
}
private string _connectionString;
/// <summary>
/// Gets or sets a value for the Umbraco database connection string..
/// </summary>
public ConfigConnectionString UmbracoConnectionString { get; set; } = new ConfigConnectionString(Constants.System.UmbracoConnectionName, null);
/// <summary>
/// The default provider name when not present in configuration.
/// </summary>
public const string DefaultProviderName = "Microsoft.Data.SqlClient";
/// <summary>
/// The DataDirectory placeholder.
/// </summary>
public const string DataDirectoryPlaceholder = "|DataDirectory|";
/// <summary>
/// The postfix used to identify a connection strings provider setting.
/// </summary>
public const string ProviderNamePostfix = "_ProviderName";
public string Name { get; set; }
public string ConnectionString
{
get => _connectionString;
set => _connectionString = value.ReplaceDataDirectoryPlaceholder();
}
public string ProviderName { get; set; } = DefaultProviderName;
}

25
src/Umbraco.Core/Configuration/Models/GlobalSettings.cs
View File

@@ -27,7 +27,8 @@ namespace Umbraco.Cms.Core.Configuration.Models
internal const bool StaticInstallMissingDatabase = false;
internal const bool StaticDisableElectionForSingleServer = false;
internal const string StaticNoNodesViewPath = "~/umbraco/UmbracoWebsite/NoNodes.cshtml";
internal const string StaticSqlWriteLockTimeOut = "00:00:05";
internal const string StaticDistributedLockingReadLockDefaultTimeout = "00:01:00";
internal const string StaticDistributedLockingWriteLockDefaultTimeout = "00:00:05";
internal const bool StaticSanitizeTinyMce = false;
internal const int StaticMainDomReleaseSignalPollingInterval = 2000;
@@ -201,12 +202,26 @@ namespace Umbraco.Cms.Core.Configuration.Models
public bool SanitizeTinyMce { get; set; } = StaticSanitizeTinyMce;
/// <summary>
/// An int value representing the time in milliseconds to lock the database for a write operation
/// Gets or sets a value representing the maximum time to wait whilst attempting to obtain a distributed read lock.
/// </summary>
/// <remarks>
/// The default value is 5000 milliseconds.
/// The default value is 60 seconds.
/// </remarks>
[DefaultValue(StaticSqlWriteLockTimeOut)]
public TimeSpan SqlWriteLockTimeOut { get; set; } = TimeSpan.Parse(StaticSqlWriteLockTimeOut);
[DefaultValue(StaticDistributedLockingReadLockDefaultTimeout)]
public TimeSpan DistributedLockingReadLockDefaultTimeout { get; set; } = TimeSpan.Parse(StaticDistributedLockingReadLockDefaultTimeout);
/// <summary>
/// Gets or sets a value representing the maximum time to wait whilst attempting to obtain a distributed write lock.
/// </summary>
/// <remarks>
/// The default value is 5 seconds.
/// </remarks>
[DefaultValue(StaticDistributedLockingWriteLockDefaultTimeout)]
public TimeSpan DistributedLockingWriteLockDefaultTimeout { get; set; } = TimeSpan.Parse(StaticDistributedLockingWriteLockDefaultTimeout);
/// <summary>
/// Gets or sets a value representing the DistributedLockingMechanism to use.
/// </summary>
public string DistributedLockingMechanism { get; set; } = string.Empty;
}
}

4
src/Umbraco.Core/Configuration/Models/Validation/GlobalSettingsValidator.cs
View File

@@ -20,7 +20,7 @@ namespace Umbraco.Cms.Core.Configuration.Models.Validation
return ValidateOptionsResult.Fail(message);
}
if (!ValidateSqlWriteLockTimeOutSetting(options.SqlWriteLockTimeOut, out var message2))
if (!ValidateSqlWriteLockTimeOutSetting(options.DistributedLockingWriteLockDefaultTimeout, out var message2))
{
return ValidateOptionsResult.Fail(message2);
}
@@ -37,7 +37,7 @@ namespace Umbraco.Cms.Core.Configuration.Models.Validation
const int maximumTimeOut = 20000;
if (configuredTimeOut.TotalMilliseconds < minimumTimeOut || configuredTimeOut.TotalMilliseconds > maximumTimeOut) // between 0.1 and 20 seconds
{
message = $"The `{Constants.Configuration.ConfigGlobal}:{nameof(GlobalSettings.SqlWriteLockTimeOut)}` setting is not between the minimum of {minimumTimeOut} ms and maximum of {maximumTimeOut} ms";
message = $"The `{Constants.Configuration.ConfigGlobal}:{nameof(GlobalSettings.DistributedLockingWriteLockDefaultTimeout)}` setting is not between the minimum of {minimumTimeOut} ms and maximum of {maximumTimeOut} ms";
return false;
}

1
src/Umbraco.Core/Constants-Configuration.cs
View File

@@ -17,6 +17,7 @@ namespace Umbraco.Cms.Core
public const string ConfigCustomErrorsPrefix = ConfigPrefix + "CustomErrors:";
public const string ConfigGlobalPrefix = ConfigPrefix + "Global:";
public const string ConfigGlobalId = ConfigGlobalPrefix + "Id";
public const string ConfigGlobalDistributedLockingMechanism = ConfigGlobalPrefix + "DistributedLockingMechanism";
public const string ConfigHostingPrefix = ConfigPrefix + "Hosting:";
public const string ConfigModelsBuilderPrefix = ConfigPrefix + "ModelsBuilder:";
public const string ConfigSecurityPrefix = ConfigPrefix + "Security:";

11
src/Umbraco.Core/Constants-DatabaseProviders.cs
View File

@@ -1,11 +0,0 @@
namespace Umbraco.Cms.Core
{
public static partial class Constants
{
public static class DatabaseProviders
{
public const string SqlCe = "System.Data.SqlServerCe.4.0";
public const string SqlServer = "Microsoft.Data.SqlClient";
}
}
}

3
src/Umbraco.Core/Constants-System.cs
View File

@@ -59,6 +59,9 @@
public const string RecycleBinMediaPathPrefix = "-1,-21,";
public const int DefaultLabelDataTypeId = -92;
public const string UmbracoDefaultDatabaseName = "Umbraco";
public const string UmbracoConnectionName = "umbracoDbDSN";
}
}

2
src/Umbraco.Core/DependencyInjection/IUmbracoBuilder.cs
View File

@@ -31,7 +31,7 @@ namespace Umbraco.Cms.Core.DependencyInjection
IProfiler Profiler { get; }
AppCaches AppCaches { get; }
TBuilder WithCollectionBuilder<TBuilder>() where TBuilder : ICollectionBuilder, new();
TBuilder WithCollectionBuilder<TBuilder>() where TBuilder : ICollectionBuilder;
void Build();
}
}

2
src/Umbraco.Core/DependencyInjection/UmbracoBuilder.Configuration.cs
View File

@@ -90,6 +90,8 @@ namespace Umbraco.Cms.Core.DependencyInjection
.AddUmbracoOptions<ContentDashboardSettings>()
.AddUmbracoOptions<HelpPageSettings>();
builder.Services.AddSingleton<IConfigureOptions<ConnectionStrings>, ConfigureConnectionStrings>();
builder.Services.Configure<RequestHandlerSettings>(options => options.MergeReplacements(builder.Config));
return builder;

19
src/Umbraco.Core/DependencyInjection/UmbracoBuilder.cs
View File

@@ -103,7 +103,7 @@ namespace Umbraco.Cms.Core.DependencyInjection
/// <typeparam name="TBuilder">The type of the collection builder.</typeparam>
/// <returns>The collection builder.</returns>
public TBuilder WithCollectionBuilder<TBuilder>()
where TBuilder : ICollectionBuilder, new()
where TBuilder : ICollectionBuilder
{
Type typeOfBuilder = typeof(TBuilder);
@@ -112,7 +112,22 @@ namespace Umbraco.Cms.Core.DependencyInjection
return (TBuilder)o;
}
var builder = new TBuilder();
TBuilder builder;
if (typeof(TBuilder).GetConstructor(Type.EmptyTypes) != null)
{
builder = Activator.CreateInstance<TBuilder>();
}
else if (typeof(TBuilder).GetConstructor(new[] { typeof(IUmbracoBuilder) }) != null)
{
// Handle those collection builders which need a reference to umbraco builder i.e. DistributedLockingCollectionBuilder.
builder = (TBuilder)Activator.CreateInstance(typeof(TBuilder), this);
}
else
{
throw new InvalidOperationException("A CollectionBuilder must have either a parameterless constructor or a constructor whose only parameter is of type IUmbracoBuilder");
}
_builders[typeOfBuilder] = builder;
return builder;
}

10
src/Umbraco.Core/DistributedLocking/DistributedLockType.cs Normal file
View File

@@ -0,0 +1,10 @@
namespace Umbraco.Cms.Core.DistributedLocking;
/// <summary>
/// Represents the type of distributed lock.
/// </summary>
public enum DistributedLockType
{
ReadLock,
WriteLock
}

26
src/Umbraco.Core/DistributedLocking/Exceptions/DistributedLockingException.cs Normal file
View File

@@ -0,0 +1,26 @@
using System;
namespace Umbraco.Cms.Core.DistributedLocking.Exceptions;
/// <summary>
/// Base class for all DistributedLockingExceptions.
/// </summary>
public class DistributedLockingException : ApplicationException
{
/// <summary>
/// Initializes a new instance of the <see cref="DistributedLockingException"/> class.
/// </summary>
public DistributedLockingException(string message)
: base(message)
{
}
/// <summary>
/// Initializes a new instance of the <see cref="DistributedLockingException"/> class.
/// </summary>
// ReSharper disable once UnusedMember.Global
public DistributedLockingException(string message, Exception innerException)
: base(message, innerException)
{
}
}

15
src/Umbraco.Core/DistributedLocking/Exceptions/DistributedLockingTimeoutException.cs Normal file
View File

@@ -0,0 +1,15 @@
namespace Umbraco.Cms.Core.DistributedLocking.Exceptions;
/// <summary>
/// Base class for all DistributedLocking timeout related exceptions.
/// </summary>
public abstract class DistributedLockingTimeoutException : DistributedLockingException
{
/// <summary>
/// Initializes a new instance of the <see cref="DistributedLockingTimeoutException"/> class.
/// </summary>
protected DistributedLockingTimeoutException(int lockId, bool isWrite)
: base($"Failed to acquire {(isWrite ? "write" : "read")} lock for id: {lockId}.")
{
}
}

15
src/Umbraco.Core/DistributedLocking/Exceptions/DistributedReadLockTimeoutException.cs Normal file
View File

@@ -0,0 +1,15 @@
namespace Umbraco.Cms.Core.DistributedLocking.Exceptions;
/// <summary>
/// Exception thrown when a read lock could not be obtained in a timely manner.
/// </summary>
public class DistributedReadLockTimeoutException : DistributedLockingTimeoutException
{
/// <summary>
/// Initializes a new instance of the <see cref="DistributedReadLockTimeoutException"/> class.
/// </summary>
public DistributedReadLockTimeoutException(int lockId)
: base(lockId, false)
{
}
}

15
src/Umbraco.Core/DistributedLocking/Exceptions/DistributedWriteLockTimeoutException.cs Normal file
View File

@@ -0,0 +1,15 @@
namespace Umbraco.Cms.Core.DistributedLocking.Exceptions;
/// <summary>
/// Exception thrown when a write lock could not be obtained in a timely manner.
/// </summary>
public class DistributedWriteLockTimeoutException : DistributedLockingTimeoutException
{
/// <summary>
/// Initializes a new instance of the <see cref="DistributedWriteLockTimeoutException"/> class.
/// </summary>
public DistributedWriteLockTimeoutException(int lockId)
: base(lockId, true)
{
}
}

19
src/Umbraco.Core/DistributedLocking/IDistributedLock.cs Normal file
View File

@@ -0,0 +1,19 @@
using System;
namespace Umbraco.Cms.Core.DistributedLocking;
/// <summary>
/// Interface representing a DistributedLock.
/// </summary>
public interface IDistributedLock : IDisposable
{
/// <summary>
/// Gets the LockId.
/// </summary>
int LockId { get; }
/// <summary>
/// Gets the DistributedLockType.
/// </summary>
DistributedLockType LockType { get; }
}

51
src/Umbraco.Core/DistributedLocking/IDistributedLockingMechanism.cs Normal file
View File

@@ -0,0 +1,51 @@
using System;
using Umbraco.Cms.Core.Configuration.Models;
using Umbraco.Cms.Core.DistributedLocking.Exceptions;
namespace Umbraco.Cms.Core.DistributedLocking;
/// <summary>
/// Represents a class responsible for managing distributed locks.
/// </summary>
/// <remarks>
/// In general the rules for distributed locks are as follows.
/// <list type="bullet">
/// <item>
/// <b>Cannot</b> obtain a write lock if a read lock exists for same lock id (except during an upgrade from reader -> writer)
/// </item>
/// <item>
/// <b>Cannot</b> obtain a write lock if a write lock exists for same lock id.
/// </item>
/// <item>
/// <b>Cannot</b> obtain a read lock if a write lock exists for same lock id.
/// </item>
/// <item>
/// <b>Can</b> obtain a read lock if a read lock exists for same lock id.
/// </item>
/// </list>
/// </remarks>
public interface IDistributedLockingMechanism
{
/// <summary>
/// Gets a value indicating whether this distributed locking mechanism can be used.
/// </summary>
bool Enabled { get; }
/// <summary>
/// Obtains a distributed read lock.
/// </summary>
/// <remarks>
/// When timeout is null, implementations should use <see cref="GlobalSettings.DistributedLockingReadLockDefaultTimeout"/>.
/// </remarks>
/// <exception cref="DistributedReadLockTimeoutException">Failed to obtain distributed read lock in time.</exception>
IDistributedLock ReadLock(int lockId, TimeSpan? obtainLockTimeout = null);
/// <summary>
/// Obtains a distributed read lock.
/// </summary>
/// <remarks>
/// When timeout is null, implementations should use <see cref="GlobalSettings.DistributedLockingWriteLockDefaultTimeout"/>.
/// </remarks>
/// <exception cref="DistributedWriteLockTimeoutException">Failed to obtain distributed write lock in time.</exception>
IDistributedLock WriteLock(int lockId, TimeSpan? obtainLockTimeout = null);
}

9
src/Umbraco.Core/DistributedLocking/IDistributedLockingMechanismFactory.cs Normal file
View File

@@ -0,0 +1,9 @@
namespace Umbraco.Cms.Core.DistributedLocking;
/// <summary>
/// Picks an appropriate IDistributedLockingMechanism when multiple are registered
/// </summary>
public interface IDistributedLockingMechanismFactory
{
IDistributedLockingMechanism DistributedLockingMechanism { get; }
}

15
src/Umbraco.Core/Extensions/ConfigConnectionStringExtensions.cs
View File

@@ -1,15 +0,0 @@
// Copyright (c) Umbraco.
// See LICENSE for more details.
using Umbraco.Cms.Core.Configuration;
namespace Umbraco.Extensions
{
public static class ConfigConnectionStringExtensions
{
public static bool IsConnectionStringConfigured(this ConfigConnectionString databaseSettings)
=> databaseSettings != null &&
!string.IsNullOrWhiteSpace(databaseSettings.ConnectionString) &&
!string.IsNullOrWhiteSpace(databaseSettings.ProviderName);
}
}

35
src/Umbraco.Core/Extensions/ConnectionStringExtensions.cs Normal file
View File

@@ -0,0 +1,35 @@
// Copyright (c) Umbraco.
// See LICENSE for more details.
using System;
using Microsoft.Extensions.Configuration;
using Umbraco.Cms.Core;
using Umbraco.Cms.Core.Configuration.Models;
namespace Umbraco.Extensions
{
public static class ConnectionStringExtensions
{
public static bool IsConnectionStringConfigured(this ConnectionStrings connectionString)
=> connectionString != null &&
!string.IsNullOrWhiteSpace(connectionString.ConnectionString) &&
!string.IsNullOrWhiteSpace(connectionString.ProviderName);
/// <summary>
/// Gets a connection string from configuration with placeholders replaced.
/// </summary>
public static string GetUmbracoConnectionString(
this IConfiguration configuration,
string connectionStringName = Constants.System.UmbracoConnectionName) =>
configuration.GetConnectionString(connectionStringName).ReplaceDataDirectoryPlaceholder();
/// <summary>
/// Replaces instances of the |DataDirectory| placeholder in a string with the value of AppDomain DataDirectory.
/// </summary>
public static string ReplaceDataDirectoryPlaceholder(this string input)
{
var dataDirectory = AppDomain.CurrentDomain.GetData("DataDirectory")?.ToString();
return input?.Replace(ConnectionStrings.DataDirectoryPlaceholder, dataDirectory);
}
}
}

8
src/Umbraco.Core/Install/Models/DatabaseModel.cs
View File

@@ -1,3 +1,4 @@
using System;
using System.Runtime.Serialization;
namespace Umbraco.Cms.Core.Install.Models
@@ -5,8 +6,11 @@ namespace Umbraco.Cms.Core.Install.Models
[DataContract(Name = "database", Namespace = "")]
public class DatabaseModel
{
[DataMember(Name = "dbType")]
public DatabaseType DatabaseType { get; set; } = DatabaseType.SqlServer;
[DataMember(Name = "databaseProviderMetadataId")]
public Guid DatabaseProviderMetadataId { get; set; }
[DataMember(Name = "providerName")]
public string ProviderName { get; set; }
[DataMember(Name = "server")]
public string Server { get; set; }

11
src/Umbraco.Core/Install/Models/DatabaseType.cs
View File

@@ -1,11 +0,0 @@
namespace Umbraco.Cms.Core.Install.Models
{
public enum DatabaseType
{
SqlLocalDb,
SqlCe,
SqlServer,
SqlAzure,
Custom
}
}

12
src/Umbraco.Core/Persistence/Constants-DbProviderNames.cs
View File

@@ -1,12 +0,0 @@
 // ReSharper disable once CheckNamespace
namespace Umbraco.Cms.Core
{
static partial class Constants
{
public static class DbProviderNames
{
public const string SqlServer = "Microsoft.Data.SqlClient";
public const string SqlCe = "System.Data.SqlServerCe.4.0";
}
}
}

5
src/Umbraco.Infrastructure/Configuration/JsonConfigManipulator.cs
View File

@@ -7,6 +7,7 @@ using Microsoft.Extensions.FileProviders;
using Microsoft.Extensions.Logging;
using Newtonsoft.Json;
using Newtonsoft.Json.Linq;
using Umbraco.Cms.Core.Configuration.Models;
using Umbraco.Cms.Web.Common.DependencyInjection;
namespace Umbraco.Cms.Core.Configuration
@@ -182,8 +183,10 @@ namespace Umbraco.Cms.Core.Configuration
writer.WriteStartObject();
writer.WritePropertyName("ConnectionStrings");
writer.WriteStartObject();
writer.WritePropertyName(Cms.Core.Constants.System.UmbracoConnectionName);
writer.WritePropertyName(Constants.System.UmbracoConnectionName);
writer.WriteValue(connectionString);
writer.WritePropertyName($"{Constants.System.UmbracoConnectionName}{ConnectionStrings.ProviderNamePostfix}");
writer.WriteValue(providerName);
writer.WriteEndObject();
writer.WriteEndObject();

3
src/Umbraco.Infrastructure/DependencyInjection/UmbracoBuilder.CoreServices.cs
View File

@@ -9,6 +9,7 @@ using Umbraco.Cms.Core.Cache;
using Umbraco.Cms.Core.Configuration;
using Umbraco.Cms.Core.Configuration.Models;
using Umbraco.Cms.Core.DependencyInjection;
using Umbraco.Cms.Core.DistributedLocking;
using Umbraco.Cms.Core.Events;
using Umbraco.Cms.Core.Handlers;
using Umbraco.Cms.Core.HealthChecks.NotificationMethods;
@@ -36,6 +37,7 @@ using Umbraco.Cms.Core.Strings;
using Umbraco.Cms.Core.Templates;
using Umbraco.Cms.Core.Trees;
using Umbraco.Cms.Core.Web;
using Umbraco.Cms.Infrastructure.DistributedLocking;
using Umbraco.Cms.Infrastructure.Examine;
using Umbraco.Cms.Infrastructure.HealthChecks;
using Umbraco.Cms.Infrastructure.HostedServices;
@@ -68,6 +70,7 @@ namespace Umbraco.Cms.Infrastructure.DependencyInjection
.AddMainDom()
.AddLogging();
builder.Services.AddSingleton<IDistributedLockingMechanismFactory, DefaultDistributedLockingMechanismFactory>();
builder.Services.AddSingleton<IUmbracoDatabaseFactory, UmbracoDatabaseFactory>();
builder.Services.AddSingleton(factory => factory.GetRequiredService<IUmbracoDatabaseFactory>().SqlContext);
builder.NPocoMappers().Add<NullableDateMapper>();

64
src/Umbraco.Infrastructure/DistributedLocking/DefaultDistributedLockingMechanismFactory.cs Normal file
View File

@@ -0,0 +1,64 @@
using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading;
using Microsoft.Extensions.Options;
using Umbraco.Cms.Core.Configuration.Models;
using Umbraco.Cms.Core.DistributedLocking;
namespace Umbraco.Cms.Infrastructure.DistributedLocking;
public class DefaultDistributedLockingMechanismFactory : IDistributedLockingMechanismFactory
{
private object _lock = new();
private bool _initialized;
private IDistributedLockingMechanism _distributedLockingMechanism;
private readonly IOptionsMonitor<GlobalSettings> _globalSettings;
private readonly IEnumerable<IDistributedLockingMechanism> _distributedLockingMechanisms;
public DefaultDistributedLockingMechanismFactory(
IOptionsMonitor<GlobalSettings> globalSettings,
IEnumerable<IDistributedLockingMechanism> distributedLockingMechanisms)
{
_globalSettings = globalSettings;
_distributedLockingMechanisms = distributedLockingMechanisms;
}
public IDistributedLockingMechanism DistributedLockingMechanism
{
get
{
EnsureInitialized();
return _distributedLockingMechanism;
}
}
private void EnsureInitialized()
=> LazyInitializer.EnsureInitialized(ref _distributedLockingMechanism, ref _initialized, ref _lock, Initialize);
private IDistributedLockingMechanism Initialize()
{
var configured = _globalSettings.CurrentValue.DistributedLockingMechanism;
if (!string.IsNullOrEmpty(configured))
{
IDistributedLockingMechanism value = _distributedLockingMechanisms
.FirstOrDefault(x => x.GetType().FullName?.EndsWith(configured) ?? false);
if (value == null)
{
throw new InvalidOperationException($"Couldn't find DistributedLockingMechanism specified by global config: {configured}");
}
}
IDistributedLockingMechanism defaultMechanism = _distributedLockingMechanisms.FirstOrDefault(x => x.Enabled);
if (defaultMechanism != null)
{
return defaultMechanism;
}
throw new InvalidOperationException($"Couldn't find an appropriate default distributed locking mechanism.");
}
}

9
src/Umbraco.Infrastructure/Install/InstallHelper.cs
View File

@@ -95,9 +95,10 @@ namespace Umbraco.Cms.Infrastructure.Install
/// <value>
/// <c>true</c> if this is a brand new install; otherwise, <c>false</c>.
/// </value>
private bool IsBrandNewInstall => _connectionStrings.CurrentValue.UmbracoConnectionString?.IsConnectionStringConfigured() != true ||
_databaseBuilder.IsDatabaseConfigured == false ||
_databaseBuilder.CanConnectToDatabase == false ||
_databaseBuilder.IsUmbracoInstalled() == false;
private bool IsBrandNewInstall =>
_connectionStrings.Get(Constants.System.UmbracoConnectionName).IsConnectionStringConfigured() == false ||
_databaseBuilder.IsDatabaseConfigured == false ||
_databaseBuilder.CanConnectToDatabase == false ||
_databaseBuilder.IsUmbracoInstalled() == false;
}
}

93
src/Umbraco.Infrastructure/Install/InstallSteps/DatabaseConfigureStep.cs
View File

@@ -1,6 +1,6 @@
using System;
using System.Collections.Generic;
using System.Runtime.InteropServices;
using System.Linq;
using System.Threading.Tasks;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Options;
@@ -20,116 +20,55 @@ namespace Umbraco.Cms.Infrastructure.Install.InstallSteps
{
private readonly DatabaseBuilder _databaseBuilder;
private readonly ILogger<DatabaseConfigureStep> _logger;
private readonly IEnumerable<IDatabaseProviderMetadata> _databaseProviderMetadata;
private readonly IOptionsMonitor<ConnectionStrings> _connectionStrings;
public DatabaseConfigureStep(DatabaseBuilder databaseBuilder, IOptionsMonitor<ConnectionStrings> connectionStrings, ILogger<DatabaseConfigureStep> logger)
public DatabaseConfigureStep(
DatabaseBuilder databaseBuilder,
IOptionsMonitor<ConnectionStrings> connectionStrings,
ILogger<DatabaseConfigureStep> logger,
IEnumerable<IDatabaseProviderMetadata> databaseProviderMetadata)
{
_databaseBuilder = databaseBuilder;
_connectionStrings = connectionStrings;
_logger = logger;
_databaseProviderMetadata = databaseProviderMetadata;
}
public override Task<InstallSetupResult> ExecuteAsync(DatabaseModel database)
public override Task<InstallSetupResult> ExecuteAsync(DatabaseModel databaseSettings)
{
//if the database model is null then we will apply the defaults
if (database == null)
{
database = new DatabaseModel();
if (IsLocalDbAvailable())
{
database.DatabaseType = DatabaseType.SqlLocalDb;
}
else if (IsSqlCeAvailable())
{
database.DatabaseType = DatabaseType.SqlCe;
}
}
if (_databaseBuilder.CanConnect(database.DatabaseType.ToString(), database.ConnectionString, database.Server, database.DatabaseName, database.Login, database.Password, database.IntegratedAuth) == false)
if (!_databaseBuilder.ConfigureDatabaseConnection(databaseSettings, isTrialRun: false))
{
throw new InstallException("Could not connect to the database");
}
ConfigureConnection(database);
return Task.FromResult<InstallSetupResult>(null);
}
private void ConfigureConnection(DatabaseModel database)
{
if (database.ConnectionString.IsNullOrWhiteSpace() == false)
{
_databaseBuilder.ConfigureDatabaseConnection(database.ConnectionString);
}
else if (database.DatabaseType == DatabaseType.SqlLocalDb)
{
_databaseBuilder.ConfigureSqlLocalDbDatabaseConnection();
}
else if (database.DatabaseType == DatabaseType.SqlCe)
{
_databaseBuilder.ConfigureEmbeddedDatabaseConnection();
}
else if (database.IntegratedAuth)
{
_databaseBuilder.ConfigureIntegratedSecurityDatabaseConnection(database.Server, database.DatabaseName);
}
else
{
var password = database.Password.Replace("'", "''");
password = string.Format("'{0}'", password);
_databaseBuilder.ConfigureDatabaseConnection(database.Server, database.DatabaseName, database.Login, password, database.DatabaseType.ToString());
}
}
public override object ViewModel
{
get
{
var databases = new List<object>()
{
new { name = "Microsoft SQL Server", id = DatabaseType.SqlServer.ToString() },
new { name = "Microsoft SQL Azure", id = DatabaseType.SqlAzure.ToString() },
new { name = "Custom connection string", id = DatabaseType.Custom.ToString() },
};
if (IsSqlCeAvailable())
{
databases.Insert(0, new { name = "Microsoft SQL Server Compact (SQL CE)", id = DatabaseType.SqlCe.ToString() });
}
if (IsLocalDbAvailable())
{
// Ensure this is always inserted as first when available
databases.Insert(0, new { name = "Microsoft SQL Server Express (LocalDB)", id = DatabaseType.SqlLocalDb.ToString() });
}
var options = _databaseProviderMetadata
.Where(x => x.IsAvailable)
.OrderBy(x => x.SortOrder)
.ToList();
return new
{
databases
databases = options
};
}
}
public static bool IsLocalDbAvailable() => new LocalDb().IsAvailable;
public static bool IsSqlCeAvailable() =>
// NOTE: Type.GetType will only return types that are currently loaded into the appdomain. In this case
// that is ok because we know if this is availalbe we will have manually loaded it into the appdomain.
// Else we'd have to use Assembly.LoadFrom and need to know the DLL location here which we don't need to do.
RuntimeInformation.IsOSPlatform(OSPlatform.Windows) &&
!(Type.GetType("Umbraco.Cms.Persistence.SqlCe.SqlCeSyntaxProvider, Umbraco.Persistence.SqlCe") is null);
public override string View => ShouldDisplayView() ? base.View : "";
public override bool RequiresExecution(DatabaseModel model) => ShouldDisplayView();
private bool ShouldDisplayView()
{
//If the connection string is already present in web.config we don't need to show the settings page and we jump to installing/upgrading.
var databaseSettings = _connectionStrings.CurrentValue.UmbracoConnectionString;
var databaseSettings = _connectionStrings.Get(Core.Constants.System.UmbracoConnectionName);
if (databaseSettings.IsConnectionStringConfigured())
{

2
src/Umbraco.Infrastructure/Install/InstallSteps/DatabaseUpgradeStep.cs
View File

@@ -77,7 +77,7 @@ namespace Umbraco.Cms.Infrastructure.Install.InstallSteps
return false;
}
var databaseSettings = _connectionStrings.CurrentValue.UmbracoConnectionString;
var databaseSettings = _connectionStrings.Get(Core.Constants.System.UmbracoConnectionName);
if (databaseSettings.IsConnectionStringConfigured())
{

23
src/Umbraco.Infrastructure/Install/InstallSteps/NewInstallStep.cs
View File

@@ -1,5 +1,7 @@
using System;
using System.Collections.Generic;
using System.Collections.Specialized;
using System.Linq;
using System.Net.Http;
using System.Text;
using System.Threading.Tasks;
@@ -37,6 +39,7 @@ namespace Umbraco.Cms.Infrastructure.Install.InstallSteps
private readonly ICookieManager _cookieManager;
private readonly IBackOfficeUserManager _userManager;
private readonly IDbProviderFactoryCreator _dbProviderFactoryCreator;
private readonly IEnumerable<IDatabaseProviderMetadata> _databaseProviderMetadata;
public NewInstallStep(
IUserService userService,
@@ -47,7 +50,8 @@ namespace Umbraco.Cms.Infrastructure.Install.InstallSteps
IOptionsMonitor<ConnectionStrings> connectionStrings,
ICookieManager cookieManager,
IBackOfficeUserManager userManager,
IDbProviderFactoryCreator dbProviderFactoryCreator)
IDbProviderFactoryCreator dbProviderFactoryCreator,
IEnumerable<IDatabaseProviderMetadata> databaseProviderMetadata)
{
_userService = userService ?? throw new ArgumentNullException(nameof(userService));
_databaseBuilder = databaseBuilder ?? throw new ArgumentNullException(nameof(databaseBuilder));
@@ -58,6 +62,7 @@ namespace Umbraco.Cms.Infrastructure.Install.InstallSteps
_cookieManager = cookieManager;
_userManager = userManager ?? throw new ArgumentNullException(nameof(userManager));
_dbProviderFactoryCreator = dbProviderFactoryCreator ?? throw new ArgumentNullException(nameof(dbProviderFactoryCreator));
_databaseProviderMetadata = databaseProviderMetadata;
}
public override async Task<InstallSetupResult> ExecuteAsync(UserModel user)
@@ -113,11 +118,22 @@ namespace Umbraco.Cms.Infrastructure.Install.InstallSteps
{
get
{
var quickInstallSettings = _databaseProviderMetadata
.Where(x => x.SupportsQuickInstall)
.Where(x => x.IsAvailable)
.OrderBy(x => x.SortOrder)
.Select(x => new
{
displayName = x.DisplayName,
defaultDatabaseName = x.DefaultDatabaseName,
})
.FirstOrDefault();
return new
{
minCharLength = _passwordConfiguration.RequiredLength,
minNonAlphaNumericLength = _passwordConfiguration.GetMinNonAlphaNumericChars(),
quickInstallAvailable = DatabaseConfigureStep.IsSqlCeAvailable() || DatabaseConfigureStep.IsLocalDbAvailable(),
quickInstallSettings,
customInstallAvailable = !GetInstallState().HasFlag(InstallState.ConnectionStringConfigured)
};
}
@@ -139,10 +155,11 @@ namespace Umbraco.Cms.Infrastructure.Install.InstallSteps
{
var installState = InstallState.Unknown;
// TODO: we need to do a null check here since this could be entirely missing and we end up with a null ref
// exception in the installer.
var databaseSettings = _connectionStrings.CurrentValue.UmbracoConnectionString;
var databaseSettings = _connectionStrings.Get(Constants.System.UmbracoConnectionName);
var hasConnString = databaseSettings != null && _databaseBuilder.IsDatabaseConfigured;
if (hasConnString)

11
src/Umbraco.Infrastructure/Migrations/Expressions/Create/Table/CreateTableOfDtoBuilder.cs
View File

@@ -29,15 +29,8 @@ namespace Umbraco.Cms.Infrastructure.Migrations.Expressions.Create.Table
var syntax = _context.SqlContext.SqlSyntax;
var tableDefinition = DefinitionFactory.GetTableDefinition(TypeOfDto, syntax);
ExecuteSql(syntax.Format(tableDefinition));
if (WithoutKeysAndIndexes)
return;
ExecuteSql(syntax.FormatPrimaryKey(tableDefinition));
foreach (var sql in syntax.Format(tableDefinition.ForeignKeys))
ExecuteSql(sql);
foreach (var sql in syntax.Format(tableDefinition.Indexes))
ExecuteSql(sql);
syntax.HandleCreateTable(_context.Database, tableDefinition, WithoutKeysAndIndexes);
_context.BuildingExpression = false;
}
private void ExecuteSql(string sql)

15
src/Umbraco.Infrastructure/Migrations/Expressions/Delete/KeysAndIndexes/DeleteKeysAndIndexesBuilder.cs
View File

@@ -6,6 +6,21 @@ using Umbraco.Extensions;
namespace Umbraco.Cms.Infrastructure.Migrations.Expressions.Delete.KeysAndIndexes
{
/// <remarks>
/// <para>
/// Assuming we stick with the current migrations setup this will need to be altered to
/// delegate to SQL syntax provider (we can drop indexes but not PK/FK).
/// </para>
/// <para>
/// 1. For SQLite, rename table.<br/>
/// 2. Create new table with expected keys.<br/>
/// 3. Insert into new from renamed<br/>
/// 4. Drop renamed.<br/>
/// </para>
/// <para>
/// Read more <a href="https://www.sqlite.org/omitted.html">SQL Features That SQLite Does Not Implement</a>
/// </para>
/// </remarks>
public class DeleteKeysAndIndexesBuilder : IExecutableBuilder
{
private readonly IMigrationContext _context;

228
src/Umbraco.Infrastructure/Migrations/Install/DatabaseBuilder.cs
View File

@@ -1,9 +1,14 @@
using System;
using System.Collections.Generic;
using System.Data.Common;
using System.Linq;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Options;
using Umbraco.Cms.Core;
using Umbraco.Cms.Core.Configuration;
using Umbraco.Cms.Core.Configuration.Models;
using Umbraco.Cms.Core.Install;
using Umbraco.Cms.Core.Install.Models;
using Umbraco.Cms.Core.Migrations;
using Umbraco.Cms.Core.Scoping;
using Umbraco.Cms.Core.Services;
@@ -32,6 +37,7 @@ namespace Umbraco.Cms.Infrastructure.Migrations.Install
private readonly IOptionsMonitor<ConnectionStrings> _connectionStrings;
private readonly IMigrationPlanExecutor _migrationPlanExecutor;
private readonly DatabaseSchemaCreatorFactory _databaseSchemaCreatorFactory;
private readonly IEnumerable<IDatabaseProviderMetadata> _databaseProviderMetadata;
private DatabaseSchemaResult _databaseSchemaValidationResult;
@@ -50,7 +56,8 @@ namespace Umbraco.Cms.Infrastructure.Migrations.Install
IOptionsMonitor<GlobalSettings> globalSettings,
IOptionsMonitor<ConnectionStrings> connectionStrings,
IMigrationPlanExecutor migrationPlanExecutor,
DatabaseSchemaCreatorFactory databaseSchemaCreatorFactory)
DatabaseSchemaCreatorFactory databaseSchemaCreatorFactory,
IEnumerable<IDatabaseProviderMetadata> databaseProviderMetadata)
{
_scopeProvider = scopeProvider;
_scopeAccessor = scopeAccessor;
@@ -64,6 +71,7 @@ namespace Umbraco.Cms.Infrastructure.Migrations.Install
_connectionStrings = connectionStrings;
_migrationPlanExecutor = migrationPlanExecutor;
_databaseSchemaCreatorFactory = databaseSchemaCreatorFactory;
_databaseProviderMetadata = databaseProviderMetadata;
}
#region Status
@@ -83,32 +91,9 @@ namespace Umbraco.Cms.Infrastructure.Migrations.Install
/// <summary>
/// Verifies whether a it is possible to connect to a database.
/// </summary>
public bool CanConnect(string databaseType, string connectionString, string server, string database, string login, string password, bool integratedAuth)
public bool CanConnect(string connectionString, string providerName)
{
// we do not test SqlCE or LocalDB connections
if (databaseType.InvariantContains("SqlCe") || databaseType.InvariantContains("SqlLocalDb"))
return true;
string providerName;
if (string.IsNullOrWhiteSpace(connectionString) == false)
{
providerName = ConfigConnectionString.ParseProviderName(connectionString);
}
else if (integratedAuth)
{
// has to be Sql Server
providerName = Constants.DbProviderNames.SqlServer;
connectionString = GetIntegratedSecurityDatabaseConnectionString(server, database);
}
else
{
connectionString = GetDatabaseConnectionString(
server, database, login, password,
databaseType, out providerName);
}
var factory = _dbProviderFactoryCreator.CreateFactory(providerName);
DbProviderFactory factory = _dbProviderFactoryCreator.CreateFactory(providerName);
return DbConnectionExtensions.IsConnectionAvailable(connectionString, factory);
}
@@ -147,65 +132,60 @@ namespace Umbraco.Cms.Infrastructure.Migrations.Install
#region Configure Connection String
public const string EmbeddedDatabaseConnectionString = @"Data Source=|DataDirectory|\Umbraco.sdf;Flush Interval=1";
/// <summary>
/// Configures a connection string for the embedded database.
/// </summary>
public void ConfigureEmbeddedDatabaseConnection()
public bool ConfigureDatabaseConnection(DatabaseModel databaseSettings, bool isTrialRun)
{
const string connectionString = EmbeddedDatabaseConnectionString;
const string providerName = Constants.DbProviderNames.SqlCe;
IDatabaseProviderMetadata providerMeta;
_configManipulator.SaveConnectionString(connectionString, providerName);
Configure(connectionString, providerName, true);
// if the database model is null then we will attempt quick install.
if (databaseSettings == null)
{
providerMeta = _databaseProviderMetadata
.OrderBy(x => x.SortOrder)
.Where(x => x.SupportsQuickInstall)
.FirstOrDefault(x => x.IsAvailable);
databaseSettings = new DatabaseModel
{
DatabaseName = providerMeta?.DefaultDatabaseName,
};
}
else
{
providerMeta = _databaseProviderMetadata
.FirstOrDefault(x => x.Id == databaseSettings.DatabaseProviderMetadataId);
}
if (providerMeta == null)
{
throw new InstallException("Unable to determine database provider configuration.");
}
var connectionString = providerMeta.GenerateConnectionString(databaseSettings);
var providerName = databaseSettings.ProviderName ?? providerMeta.ProviderName;
if (providerMeta.RequiresConnectionTest && !CanConnect(connectionString, providerName))
{
return false;
}
if (!isTrialRun)
{
_configManipulator.SaveConnectionString(connectionString, providerName);
Configure(connectionString, providerName, _globalSettings.CurrentValue.InstallMissingDatabase || providerMeta.ForceCreateDatabase);
}
return true;
}
public const string LocalDbConnectionString = @"Data Source=(localdb)\MSSQLLocalDB;AttachDbFilename=|DataDirectory|\Umbraco.mdf;Integrated Security=True";
public void ConfigureSqlLocalDbDatabaseConnection()
{
string connectionString = LocalDbConnectionString;
const string providerName = Constants.DbProviderNames.SqlServer;
_configManipulator.SaveConnectionString(connectionString, providerName);
Configure(connectionString, providerName, true);
}
/// <summary>
/// Configures a connection string that has been entered manually.
/// </summary>
/// <param name="connectionString">A connection string.</param>
/// <remarks>Has to be SQL Server</remarks>
public void ConfigureDatabaseConnection(string connectionString)
{
_configManipulator.SaveConnectionString(connectionString, null);
Configure(connectionString, null, _globalSettings.CurrentValue.InstallMissingDatabase);
}
/// <summary>
/// Configures a connection string from the installer.
/// </summary>
/// <param name="server">The name or address of the database server.</param>
/// <param name="databaseName">The name of the database.</param>
/// <param name="user">The user name.</param>
/// <param name="password">The user password.</param>
/// <param name="databaseProvider">The name of the provider (Sql, Sql Azure, Sql Ce).</param>
public void ConfigureDatabaseConnection(string server, string databaseName, string user, string password, string databaseProvider)
{
var connectionString = GetDatabaseConnectionString(server, databaseName, user, password, databaseProvider, out var providerName);
_configManipulator.SaveConnectionString(connectionString, providerName);
Configure(connectionString, providerName, _globalSettings.CurrentValue.InstallMissingDatabase);
}
private void Configure(string connectionString, string providerName, bool installMissingDatabase)
{
// Update existing connection string
var umbracoConnectionString = new ConfigConnectionString(Constants.System.UmbracoConnectionName, connectionString, providerName);
_connectionStrings.CurrentValue.UmbracoConnectionString = umbracoConnectionString;
var umbracoConnectionString = _connectionStrings.Get(Core.Constants.System.UmbracoConnectionName);
umbracoConnectionString.ConnectionString = connectionString;
umbracoConnectionString.ProviderName = providerName;
_databaseFactory.Configure(umbracoConnectionString.ConnectionString, umbracoConnectionString.ProviderName);
_databaseFactory.Configure(umbracoConnectionString);
if (installMissingDatabase)
{
@@ -213,102 +193,6 @@ namespace Umbraco.Cms.Infrastructure.Migrations.Install
}
}
/// <summary>
/// Gets a connection string from the installer.
/// </summary>
/// <param name="server">The name or address of the database server.</param>
/// <param name="databaseName">The name of the database.</param>
/// <param name="user">The user name.</param>
/// <param name="password">The user password.</param>
/// <param name="databaseProvider">The name of the provider (Sql, Sql Azure, Sql Ce).</param>
/// <param name="providerName"></param>
/// <returns>A connection string.</returns>
public static string GetDatabaseConnectionString(string server, string databaseName, string user, string password, string databaseProvider, out string providerName)
{
providerName = Constants.DbProviderNames.SqlServer;
if (databaseProvider.InvariantContains("Azure"))
return GetAzureConnectionString(server, databaseName, user, password);
return $"server={server};database={databaseName};user id={user};password={password}";
}
/// <summary>
/// Configures a connection string using Microsoft SQL Server integrated security.
/// </summary>
/// <param name="server">The name or address of the database server.</param>
/// <param name="databaseName">The name of the database</param>
public void ConfigureIntegratedSecurityDatabaseConnection(string server, string databaseName)
{
var connectionString = GetIntegratedSecurityDatabaseConnectionString(server, databaseName);
const string providerName = Constants.DbProviderNames.SqlServer;
_configManipulator.SaveConnectionString(connectionString, providerName);
_databaseFactory.Configure(connectionString, providerName);
}
/// <summary>
/// Gets a connection string using Microsoft SQL Server integrated security.
/// </summary>
/// <param name="server">The name or address of the database server.</param>
/// <param name="databaseName">The name of the database</param>
/// <returns>A connection string.</returns>
public static string GetIntegratedSecurityDatabaseConnectionString(string server, string databaseName)
{
return $"Server={server};Database={databaseName};Integrated Security=true";
}
/// <summary>
/// Gets an Azure connection string.
/// </summary>
/// <param name="server">The name or address of the database server.</param>
/// <param name="databaseName">The name of the database.</param>
/// <param name="user">The user name.</param>
/// <param name="password">The user password.</param>
/// <returns>A connection string.</returns>
public static string GetAzureConnectionString(string server, string databaseName, string user, string password)
{
if (server.Contains(".") && ServerStartsWithTcp(server) == false)
server = $"tcp:{server}";
if (server.Contains(".") == false && ServerStartsWithTcp(server))
{
string serverName = server.Contains(",")
? server.Substring(0, server.IndexOf(",", StringComparison.Ordinal))
: server;
var portAddition = string.Empty;
if (server.Contains(","))
portAddition = server.Substring(server.IndexOf(",", StringComparison.Ordinal));
server = $"{serverName}.database.windows.net{portAddition}";
}
if (ServerStartsWithTcp(server) == false)
server = $"tcp:{server}.database.windows.net";
if (server.Contains(",") == false)
server = $"{server},1433";
if (user.Contains("@") == false)
{
var userDomain = server;
if (ServerStartsWithTcp(server))
userDomain = userDomain.Substring(userDomain.IndexOf(":", StringComparison.Ordinal) + 1);
if (userDomain.Contains("."))
userDomain = userDomain.Substring(0, userDomain.IndexOf(".", StringComparison.Ordinal));
user = $"{user}@{userDomain}";
}
return $"Server={server};Database={databaseName};User ID={user};Password={password}";
}
private static bool ServerStartsWithTcp(string server) => server.InvariantStartsWith("tcp:");
#endregion
#region Database Schema

2
src/Umbraco.Infrastructure/Migrations/Install/DatabaseDataCreator.cs
View File

@@ -81,7 +81,7 @@ namespace Umbraco.Cms.Infrastructure.Migrations.Install
if (tableName.Equals(Cms.Core.Constants.DatabaseSchema.Tables.LogViewerQuery))
CreateLogViewerQueryData();
_logger.LogInformation("Done creating table {TableName} data.", tableName);
_logger.LogInformation("Completed creating data in {TableName}", tableName);
}
private void CreateNodeData()

42
src/Umbraco.Infrastructure/Migrations/Install/DatabaseSchemaCreator.cs
View File

@@ -89,8 +89,12 @@ namespace Umbraco.Cms.Infrastructure.Migrations.Install
private readonly ILoggerFactory _loggerFactory;
private readonly IUmbracoVersion _umbracoVersion;
public DatabaseSchemaCreator(IUmbracoDatabase database, ILogger<DatabaseSchemaCreator> logger,
ILoggerFactory loggerFactory, IUmbracoVersion umbracoVersion, IEventAggregator eventAggregator)
public DatabaseSchemaCreator(
IUmbracoDatabase database,
ILogger<DatabaseSchemaCreator> logger,
ILoggerFactory loggerFactory,
IUmbracoVersion umbracoVersion,
IEventAggregator eventAggregator)
{
_database = database ?? throw new ArgumentNullException(nameof(database));
_logger = logger ?? throw new ArgumentNullException(nameof(logger));
@@ -153,8 +157,7 @@ namespace Umbraco.Cms.Infrastructure.Migrations.Install
if (creatingNotification.Cancel == false)
{
var dataCreation = new DatabaseDataCreator(_database,
_loggerFactory.CreateLogger<DatabaseDataCreator>(), _umbracoVersion);
var dataCreation = new DatabaseDataCreator(_database, _loggerFactory.CreateLogger<DatabaseDataCreator>(), _umbracoVersion);
foreach (Type table in OrderedTables)
{
CreateTable(false, table, dataCreation);
@@ -448,12 +451,6 @@ namespace Umbraco.Cms.Infrastructure.Migrations.Install
TableDefinition tableDefinition = DefinitionFactory.GetTableDefinition(modelType, SqlSyntax);
var tableName = tableDefinition.Name;
var createSql = SqlSyntax.Format(tableDefinition);
var createPrimaryKeySql = SqlSyntax.FormatPrimaryKey(tableDefinition);
List<string> foreignSql = SqlSyntax.Format(tableDefinition.ForeignKeys);
List<string> indexSql = SqlSyntax.Format(tableDefinition.Indexes);
var tableExist = TableExists(tableName);
if (overwrite && tableExist)
{
@@ -471,18 +468,11 @@ namespace Umbraco.Cms.Infrastructure.Migrations.Install
}
//Execute the Create Table sql
_database.Execute(new Sql(createSql));
_logger.LogInformation("Create Table {TableName}: \n {Sql}", tableName, createSql);
//If any statements exists for the primary key execute them here
if (string.IsNullOrEmpty(createPrimaryKeySql) == false)
{
_database.Execute(new Sql(createPrimaryKeySql));
_logger.LogInformation("Create Primary Key:\n {Sql}", createPrimaryKeySql);
}
SqlSyntax.HandleCreateTable(_database, tableDefinition);
if (SqlSyntax.SupportsIdentityInsert() && tableDefinition.Columns.Any(x => x.IsIdentity))
{
// This should probably delegate to whole thing to the syntax provider
_database.Execute(new Sql($"SET IDENTITY_INSERT {SqlSyntax.GetQuotedTableName(tableName)} ON "));
}
@@ -496,20 +486,6 @@ namespace Umbraco.Cms.Infrastructure.Migrations.Install
_database.Execute(new Sql($"SET IDENTITY_INSERT {SqlSyntax.GetQuotedTableName(tableName)} OFF;"));
}
//Loop through index statements and execute sql
foreach (var sql in indexSql)
{
_database.Execute(new Sql(sql));
_logger.LogInformation("Create Index:\n {Sql}", sql);
}
//Loop through foreignkey statements and execute sql
foreach (var sql in foreignSql)
{
_database.Execute(new Sql(sql));
_logger.LogInformation("Create Foreign Key:\n {Sql}", sql);
}
if (overwrite)
{
_logger.LogInformation("Table {TableName} was recreated", tableName);

14
src/Umbraco.Infrastructure/Migrations/MigrationBase_Extra.cs
View File

@@ -86,18 +86,8 @@ namespace Umbraco.Cms.Infrastructure.Migrations
protected void ReplaceColumn<T>(string tableName, string currentName, string newName)
{
if (DatabaseType.IsSqlCe())
{
AddColumn<T>(tableName, newName, out var sqls);
Execute.Sql($"UPDATE {SqlSyntax.GetQuotedTableName(tableName)} SET {SqlSyntax.GetQuotedColumnName(newName)}={SqlSyntax.GetQuotedColumnName(currentName)}").Do();
foreach (var sql in sqls) Execute.Sql(sql).Do();
Delete.Column(currentName).FromTable(tableName).Do();
}
else
{
Execute.Sql(SqlSyntax.FormatColumnRename(tableName, currentName, newName)).Do();
AlterColumn<T>(tableName, newName);
}
Execute.Sql(SqlSyntax.FormatColumnRename(tableName, currentName, newName)).Do();
AlterColumn<T>(tableName, newName);
}
protected bool TableExists(string tableName)

2
src/Umbraco.Infrastructure/Migrations/MigrationExpressionBase.cs
View File

@@ -120,7 +120,7 @@ namespace Umbraco.Cms.Infrastructure.Migrations
protected void AppendStatementSeparator(StringBuilder stmtBuilder)
{
stmtBuilder.AppendLine(";");
if (DatabaseType.IsSqlServerOrCe())
if (DatabaseType.IsSqlServer())
stmtBuilder.AppendLine("GO");
}

16
src/Umbraco.Infrastructure/Migrations/Upgrade/V_8_0_0/RenameMediaVersionTable.cs
View File

@@ -18,24 +18,12 @@ namespace Umbraco.Cms.Infrastructure.Migrations.Upgrade.V_8_0_0
AddColumn<MediaVersionDto>("id", out var sqls);
if (Database.DatabaseType.IsSqlCe())
{
// SQLCE does not support UPDATE...FROM
var versions = Database.Fetch<dynamic>($@"SELECT v.versionId, v.id
FROM cmsContentVersion v
JOIN umbracoNode n on v.contentId=n.id
WHERE n.nodeObjectType='{Cms.Core.Constants.ObjectTypes.Media}'");
foreach (var t in versions)
Execute.Sql($"UPDATE {Cms.Core.Constants.DatabaseSchema.Tables.MediaVersion} SET id={t.id} WHERE versionId='{t.versionId}'").Do();
}
else
{
Database.Execute($@"UPDATE {Cms.Core.Constants.DatabaseSchema.Tables.MediaVersion} SET id=v.id
Database.Execute($@"UPDATE {Cms.Core.Constants.DatabaseSchema.Tables.MediaVersion} SET id=v.id
FROM {Cms.Core.Constants.DatabaseSchema.Tables.MediaVersion} m
JOIN cmsContentVersion v on m.versionId = v.versionId
JOIN umbracoNode n on v.contentId=n.id
WHERE n.nodeObjectType='{Cms.Core.Constants.ObjectTypes.Media}'");
}
foreach (var sql in sqls)
Execute.Sql(sql).Do();

42
src/Umbraco.Infrastructure/Migrations/Upgrade/V_8_0_0/VariantsMigration.cs
View File

@@ -76,19 +76,9 @@ HAVING COUNT(v2.id) <> 1").Any())
{
Alter.Table(PreTables.PropertyData).AddColumn("versionId2").AsInt32().Nullable().Do();
if (Database.DatabaseType.IsSqlCe())
{
// SQLCE does not support UPDATE...FROM
var versions = Database.Fetch<dynamic>($"SELECT id, versionId FROM {PreTables.ContentVersion}");
foreach (var t in versions)
Database.Execute($"UPDATE {PreTables.PropertyData} SET versionId2=@v2 WHERE versionId=@v1", new { v1 = t.versionId, v2 = t.id });
}
else
{
Database.Execute($@"UPDATE {PreTables.PropertyData} SET versionId2={PreTables.ContentVersion}.id
Database.Execute($@"UPDATE {PreTables.PropertyData} SET versionId2={PreTables.ContentVersion}.id
FROM {PreTables.ContentVersion}
INNER JOIN {PreTables.PropertyData} ON {PreTables.ContentVersion}.versionId = {PreTables.PropertyData}.versionId");
}
Delete.Column("versionId").FromTable(PreTables.PropertyData).Do();
ReplaceColumn<PropertyDataDto>(PreTables.PropertyData, "versionId2", "versionId");
@@ -181,40 +171,16 @@ INNER JOIN {PreTables.PropertyData} ON {PreTables.ContentVersion}.versionId = {P
ReplaceColumn<ContentVersionDto>(PreTables.ContentVersion, "ContentId", "nodeId");
// populate contentVersion text, current and userId columns for documents
if (Database.DatabaseType.IsSqlCe())
{
// SQLCE does not support UPDATE...FROM
var documents = Database.Fetch<dynamic>($"SELECT versionId, text, published, newest, documentUser FROM {PreTables.Document}");
foreach (var t in documents)
Database.Execute($@"UPDATE {PreTables.ContentVersion} SET text=@text, {SqlSyntax.GetQuotedColumnName("current")}=@current, userId=@userId WHERE versionId=@versionId",
new { text = t.text, current = t.newest && !t.published, userId = t.documentUser, versionId = t.versionId });
}
else
{
Database.Execute($@"UPDATE {PreTables.ContentVersion} SET text=d.text, {SqlSyntax.GetQuotedColumnName("current")}=(d.newest & ~d.published), userId=d.documentUser
Database.Execute($@"UPDATE {PreTables.ContentVersion} SET text=d.text, {SqlSyntax.GetQuotedColumnName("current")}=(d.newest & ~d.published), userId=d.documentUser
FROM {PreTables.ContentVersion} v INNER JOIN {PreTables.Document} d ON d.versionId = v.versionId");
}
// populate contentVersion text and current columns for non-documents, userId is default
if (Database.DatabaseType.IsSqlCe())
{
// SQLCE does not support UPDATE...FROM
var otherContent = Database.Fetch<dynamic>($@"SELECT cver.versionId, n.text
Database.Execute($@"UPDATE {PreTables.ContentVersion} SET text=n.text, {SqlSyntax.GetQuotedColumnName("current")}=1, userId=0
FROM {PreTables.ContentVersion} cver
JOIN {SqlSyntax.GetQuotedTableName(Cms.Core.Constants.DatabaseSchema.Tables.Node)} n ON cver.nodeId=n.id
WHERE cver.versionId NOT IN (SELECT versionId FROM {SqlSyntax.GetQuotedTableName(PreTables.Document)})");
foreach (var t in otherContent)
Database.Execute($@"UPDATE {PreTables.ContentVersion} SET text=@text, {SqlSyntax.GetQuotedColumnName("current")}=1, userId=0 WHERE versionId=@versionId",
new { text = t.text, versionId = t.versionId });
}
else
{
Database.Execute($@"UPDATE {PreTables.ContentVersion} SET text=n.text, {SqlSyntax.GetQuotedColumnName("current")}=1, userId=0
FROM {PreTables.ContentVersion} cver
JOIN {SqlSyntax.GetQuotedTableName(Cms.Core.Constants.DatabaseSchema.Tables.Node)} n ON cver.nodeId=n.id
WHERE cver.versionId NOT IN (SELECT versionId FROM {SqlSyntax.GetQuotedTableName(PreTables.Document)})");
}
// create table
Create.Table<DocumentVersionDto>(withoutKeysAndIndexes: true).Do();

17
src/Umbraco.Infrastructure/Migrations/Upgrade/V_8_15_0/AddCmsContentNuByteColumn.cs
View File

@@ -17,22 +17,7 @@ namespace Umbraco.Cms.Infrastructure.Migrations.Upgrade.V_8_15_0
protected override void Migrate()
{
// allow null for the `data` field
if (DatabaseType.IsSqlCe())
{
// SQLCE does not support altering NTEXT, so we have to jump through some hoops to do it
// All column ordering must remain the same as what is defined in the DTO so we need to create a temp table,
// drop orig and then re-create/copy.
Create.Table<ContentNuDtoTemp>(withoutKeysAndIndexes: true).Do();
Execute.Sql($"INSERT INTO [{TempTableName}] SELECT nodeId, published, data, rv FROM [{Constants.DatabaseSchema.Tables.NodeData}]").Do();
Delete.Table(Constants.DatabaseSchema.Tables.NodeData).Do();
Create.Table<ContentNuDto>().Do();
Execute.Sql($"INSERT INTO [{Constants.DatabaseSchema.Tables.NodeData}] SELECT nodeId, published, data, rv, NULL FROM [{TempTableName}]").Do();
}
else
{
AlterColumn<ContentNuDto>(Constants.DatabaseSchema.Tables.NodeData, "data");
}
AlterColumn<ContentNuDto>(Constants.DatabaseSchema.Tables.NodeData, "data");
var columns = SqlSyntax.GetColumnsInSchema(Context.Database).ToList();
AddColumnIfNotExists<ContentNuDto>(columns, "dataRaw");

5
src/Umbraco.Infrastructure/Migrations/Upgrade/V_8_15_0/UpdateCmsPropertyGroupIdSeed.cs
View File

@@ -10,10 +10,7 @@ namespace Umbraco.Cms.Infrastructure.Migrations.Upgrade.V_8_15_0
protected override void Migrate()
{
if (DatabaseType.IsSqlCe())
{
Database.Execute(Sql("ALTER TABLE [cmsPropertyTypeGroup] ALTER COLUMN [id] IDENTITY (56,1)"));
}
// NOOP - was sql ce only
}
}
}

16
src/Umbraco.Infrastructure/Migrations/Upgrade/V_9_0_0/DictionaryTablesIndexes.cs
View File

@@ -109,22 +109,6 @@ namespace Umbraco.Cms.Infrastructure.Migrations.Upgrade.V_9_0_0
{
string columnSpecification;
// If using SQL CE, we don't have access to COUNT (DISTINCT *) or CONCAT, so will need to do this by querying all records.
if (DatabaseType.IsSqlCe())
{
columnSpecification = columns.Length == 1
? StringConvertedAndQuotedColumnName(columns[0])
: $"{string.Join(" + ", columns.Select(x => StringConvertedAndQuotedColumnName(x)))}";
var allRecordsQuery = Database.SqlContext.Sql()
.Select(columnSpecification)
.From<TDto>();
var allRecords = Database.Fetch<string>(allRecordsQuery);
return allRecords.Distinct().Count();
}
columnSpecification = columns.Length == 1
? QuoteColumnName(columns[0])
: $"CONCAT({string.Join(",", columns.Select(QuoteColumnName))})";

18
src/Umbraco.Infrastructure/Migrations/Upgrade/V_9_3_0/UpdateExternalLoginToUseKeyInsteadOfId.cs
View File

@@ -35,22 +35,8 @@ namespace Umbraco.Cms.Infrastructure.Migrations.Upgrade.V_9_3_0
//special trick to add the column without constraints and return the sql to add them later
AddColumn<ExternalLoginDto>("userOrMemberKey", out var sqls);
if (DatabaseType.IsSqlCe())
{
var userIds = Database.Fetch<int>(Sql().Select("userId").From<ExternalLoginDto>());
foreach (int userId in userIds)
{
Execute.Sql($"UPDATE {ExternalLoginDto.TableName} SET userOrMemberKey = '{userId.ToGuid()}' WHERE userId = {userId}").Do();
}
}
else
{
//populate the new columns with the userId as a Guid. Same method as IntExtensions.ToGuid.
Execute.Sql($"UPDATE {ExternalLoginDto.TableName} SET userOrMemberKey = CAST(CONVERT(char(8), CONVERT(BINARY(4), userId), 2) + '-0000-0000-0000-000000000000' AS UNIQUEIDENTIFIER)").Do();
}
//populate the new columns with the userId as a Guid. Same method as IntExtensions.ToGuid.
Execute.Sql($"UPDATE {ExternalLoginDto.TableName} SET userOrMemberKey = CAST(CONVERT(char(8), CONVERT(BINARY(4), userId), 2) + '-0000-0000-0000-000000000000' AS UNIQUEIDENTIFIER)").Do();
//now apply constraints (NOT NULL) to new table
foreach (var sql in sqls) Execute.Sql(sql).Do();

36
src/Umbraco.Infrastructure/Persistence/BasicBulkSqlInsertProvider.cs
View File

@@ -1,36 +0,0 @@
using System.Collections.Generic;
using System.Linq;
using Umbraco.Extensions;
namespace Umbraco.Cms.Infrastructure.Persistence
{
/// <summary>
/// A provider that just generates insert commands
/// </summary>
public class BasicBulkSqlInsertProvider : IBulkSqlInsertProvider
{
public string ProviderName => Cms.Core.Constants.DatabaseProviders.SqlServer;
public int BulkInsertRecords<T>(IUmbracoDatabase database, IEnumerable<T> records)
{
if (!records.Any()) return 0;
return BulkInsertRecordsWithCommands(database, records.ToArray());
}
/// <summary>
/// Bulk-insert records using commands.
/// </summary>
/// <typeparam name="T">The type of the records.</typeparam>
/// <param name="database">The database.</param>
/// <param name="records">The records.</param>
/// <returns>The number of records that were inserted.</returns>
internal static int BulkInsertRecordsWithCommands<T>(IUmbracoDatabase database, T[] records)
{
foreach (var command in database.GenerateBulkInsertCommands(records))
command.ExecuteNonQuery();
return records.Length; // what else?
}
}
}

55
src/Umbraco.Infrastructure/Persistence/CustomConnectionStringDatabaseProviderMetadata.cs Normal file
View File

@@ -0,0 +1,55 @@
using System;
using System.Runtime.Serialization;
using Umbraco.Cms.Core.Install.Models;
namespace Umbraco.Cms.Infrastructure.Persistence;
/// <summary>
/// Provider metadata for custom connection string setup.
/// </summary>
[DataContract]
public class CustomConnectionStringDatabaseProviderMetadata : IDatabaseProviderMetadata
{
/// <inheritdoc />
public Guid Id => new("42c0eafd-1650-4bdb-8cf6-d226e8941698");
/// <inheritdoc />
public int SortOrder => int.MaxValue;
/// <inheritdoc />
public string DisplayName => "Custom";
/// <inheritdoc />
public string DefaultDatabaseName => string.Empty;
/// <inheritdoc />
public string ProviderName => null;
/// <inheritdoc />
public bool SupportsQuickInstall => false;
/// <inheritdoc />
public bool IsAvailable => true;
/// <inheritdoc />
public bool RequiresServer => false;
/// <inheritdoc />
public string ServerPlaceholder => null;
/// <inheritdoc />
public bool RequiresCredentials => false;
/// <inheritdoc />
public bool SupportsIntegratedAuthentication => false;
/// <inheritdoc />
public bool RequiresConnectionTest => true;
/// <inheritdoc />
public bool ForceCreateDatabase => false;
/// <inheritdoc />
public string GenerateConnectionString(DatabaseModel databaseModel)
=> databaseModel.ConnectionString;
}

2
src/Umbraco.Infrastructure/Persistence/DatabaseModelDefinitions/DefinitionFactory.cs
View File

@@ -9,7 +9,7 @@ using Umbraco.Extensions;
namespace Umbraco.Cms.Infrastructure.Persistence.DatabaseModelDefinitions
{
internal static class DefinitionFactory
public static class DefinitionFactory
{
public static TableDefinition GetTableDefinition(Type modelType, ISqlSyntaxProvider sqlSyntax)
{

Some files were not shown because too many files have changed in this diff Show More