Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 16 additions & 2 deletions .github/workflows/build-and-publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -39,17 +39,31 @@ jobs:
--health-timeout 5s
--health-retries 5

mysql:
image: mysql:8.4
env:
MYSQL_ROOT_PASSWORD: test123
MYSQL_DATABASE: coresync_test
ports:
- 3306:3306
options: >-
--health-cmd "mysqladmin ping -h 127.0.0.1 -ptest123"
--health-interval 10s
--health-timeout 5s
--health-retries 10

env:
CORE-SYNC_CONNECTION_STRING: "Server=localhost;User Id=sa;Password=CoreSync_Test123!;TrustServerCertificate=True"
CORE-SYNC_POSTGRESQL_CONNECTION_STRING: "Host=localhost;Port=5432;Database=coresync_test;Username=coresync;Password=test123"
CORE-SYNC_MYSQL_CONNECTION_STRING: "Server=localhost;Port=3306;Database=coresync_test;User ID=root;Password=test123;GuidFormat=Char36"

steps:
- uses: actions/checkout@v4

- name: Setup .NET
uses: actions/setup-dotnet@v4
with:
dotnet-version: '8.0.x'
dotnet-version: '9.0.x'

- name: Restore dependencies
run: dotnet restore src/CoreSync.sln
Expand All @@ -71,7 +85,7 @@ jobs:
- name: Setup .NET
uses: actions/setup-dotnet@v4
with:
dotnet-version: '8.0.x'
dotnet-version: '9.0.x'

- name: Extract version from tag
id: version
Expand Down
8 changes: 5 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,19 +1,19 @@
# CoreSync

**CoreSync** is a .NET library for bidirectional data synchronization between databases. It supports SQLite, SQL Server, and PostgreSQL, letting you keep multiple database instances in sync — whether they're on the same machine or communicating over HTTP.
**CoreSync** is a .NET library for bidirectional data synchronization between databases. It supports SQLite, SQL Server, PostgreSQL, and MySQL, letting you keep multiple database instances in sync — whether they're on the same machine or communicating over HTTP.

[![Build status](https://ci.appveyor.com/api/projects/status/8cloij4060cbnvfp?svg=true)](https://ci.appveyor.com/project/adospace/coresync)
[![NuGet](https://img.shields.io/nuget/v/CoreSync.svg)](https://www.nuget.org/packages/CoreSync/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)

## Key Features

- **Bidirectional sync** between any supported database combination (e.g. SQLite ↔ SQL Server, PostgreSQL ↔ SQLite, etc.)
- **Bidirectional sync** between any supported database combination (e.g. SQLite ↔ SQL Server, PostgreSQL ↔ SQLite, MySQL ↔ SQLite, etc.)
- **Automatic change tracking** — detects inserts, updates, and deletes with version-based anchors
- **Conflict resolution** — built-in strategies (Skip or ForceWrite) with per-item customization
- **Filtered sync** — synchronize a subset of data using parameterized queries
- **HTTP transport** — sync over the network with ASP.NET Core server endpoints and a resilient HTTP client (with Polly retries and MessagePack binary format)
- **Multiple providers**: SQLite, SQL Server (custom change tracking), SQL Server CT (native Change Tracking), PostgreSQL
- **Multiple providers**: SQLite, SQL Server (custom change tracking), SQL Server CT (native Change Tracking), PostgreSQL, MySQL

## How It Works

Expand Down Expand Up @@ -42,6 +42,7 @@ CoreSync uses a **version-based change tracking** approach:
| `CoreSync.SqlServer` | SQL Server | Custom trigger-based |
| `CoreSync.SqlServerCT` | SQL Server | Native Change Tracking |
| `CoreSync.PostgreSQL` | PostgreSQL | Custom trigger-based |
| `CoreSync.MySql` | MySQL | Custom trigger-based |

**HTTP packages** for remote sync over the network:

Expand Down Expand Up @@ -228,6 +229,7 @@ CoreSync (core interfaces and SyncAgent)
├── CoreSync.SqlServer (SQL Server provider - custom CT)
├── CoreSync.SqlServerCT (SQL Server provider - native CT)
├── CoreSync.PostgreSQL (PostgreSQL provider)
├── CoreSync.MySql (MySQL provider)
├── CoreSync.Http (shared HTTP types)
├── CoreSync.Http.Server (ASP.NET Core sync endpoints)
└── CoreSync.Http.Client (resilient HTTP sync client)
Expand Down
7 changes: 4 additions & 3 deletions docs/comparison-with-datasync-toolkit.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ Both CoreSync and the [Datasync Community Toolkit](https://github.com/CommunityT
| **Primary purpose** | Bidirectional database-to-database sync | Client-server offline sync (successor to Azure Mobile Apps) |
| **Architecture** | Library — direct provider-to-provider | Client-server — requires ASP.NET Core controllers |
| **Sync direction** | Bidirectional (peer-to-peer) | Bidirectional (client-server only) |
| **Server databases** | SQLite, SQL Server, PostgreSQL | SQL Server, PostgreSQL, MySQL, Cosmos DB, MongoDB, LiteDB |
| **Server databases** | SQLite, SQL Server, PostgreSQL, MySQL | SQL Server, PostgreSQL, MySQL, Cosmos DB, MongoDB, LiteDB |
| **Client databases** | Any supported provider | SQLite only |
| **Change detection** | Database triggers or SQL Server native CT | Timestamp-based (`UpdatedAt` field) |
| **Entity requirements** | No changes to your schema | Must implement `ITableData` (Id, UpdatedAt, Version, Deleted) |
Expand Down Expand Up @@ -117,13 +117,14 @@ Both libraries support per-item conflict resolution. CoreSync uses a simpler mod
| SQLite | Yes | Not recommended (timestamp precision) | Yes (only option) |
| SQL Server / Azure SQL | Yes (custom triggers + native CT) | Yes | — |
| PostgreSQL | Yes | Yes | — |
| MySQL | Yes | Yes | — |
| MySQL / MariaDB | — | Yes | — |
| Cosmos DB | — | Yes | — |
| MongoDB | — | Yes | — |
| LiteDB | — | Yes | — |
| In-Memory | — | Yes (testing) | — |

CoreSync uniquely supports **SQLite as both a source and target** for sync — not just as a client-side cache. This means you can sync SQLite-to-SQLite, SQLite-to-SQL Server, or SQLite-to-PostgreSQL in any direction.
CoreSync uniquely supports **SQLite as both a source and target** for sync — not just as a client-side cache. This means you can sync SQLite-to-SQLite, SQLite-to-SQL Server, SQLite-to-PostgreSQL, or SQLite-to-MySQL in any direction.

The Datasync Toolkit has broader server-side database support (including NoSQL options), but the client is always SQLite.

Expand Down Expand Up @@ -235,7 +236,7 @@ The Datasync Toolkit has a richer security model since it's designed as a server
CoreSync is the right choice when:

- **You need database-to-database sync** — no server API required
- **You want any-to-any database combinations** — SQLite ↔ SQL Server ↔ PostgreSQL in any direction
- **You want any-to-any database combinations** — SQLite ↔ SQL Server ↔ PostgreSQL ↔ MySQL in any direction
- **You need SQLite as a full sync participant** — not just a client cache
- **Your existing schema should stay unchanged** — no base classes or required fields
- **You need broad .NET compatibility** — .NET Standard 2.0 supports everything from .NET Framework 4.6.1 to .NET 10
Expand Down
5 changes: 3 additions & 2 deletions docs/comparison-with-debezium.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ Both CoreSync and [Debezium](https://debezium.io/) deal with change data capture
| **Language / Ecosystem** | .NET (C#) | Java (JVM / Kafka ecosystem) |
| **Sync direction** | Bidirectional | Unidirectional (source to consumers) |
| **Infrastructure required** | None beyond your app and databases | Kafka + Kafka Connect (primary mode), or standalone JVM server |
| **Supported databases** | SQLite, SQL Server, PostgreSQL | MySQL, MariaDB, PostgreSQL, SQL Server, Oracle, MongoDB, Db2, Cassandra, Spanner, Vitess, Informix |
| **Supported databases** | SQLite, SQL Server, PostgreSQL, MySQL | MySQL, MariaDB, PostgreSQL, SQL Server, Oracle, MongoDB, Db2, Cassandra, Spanner, Vitess, Informix |
| **Change detection method** | Trigger-based or SQL Server native CT | Transaction log reading (binlog, WAL, redo logs) |
| **Conflict resolution** | Built-in (Skip / ForceWrite with per-item control) | N/A (unidirectional — no conflicts) |
| **Target framework** | .NET Standard 2.0 / .NET 8.0 | Java 11+ |
Expand Down Expand Up @@ -103,6 +103,7 @@ Debezium's primary deployment (Kafka Connect) requires a running Kafka cluster,
| SQLite | Yes | No |
| SQL Server | Yes (custom triggers + native CT) | Yes |
| PostgreSQL | Yes | Yes |
| MySQL | Yes | Yes |
| MySQL / MariaDB | No | Yes |
| Oracle | No | Yes |
| MongoDB | No | Yes |
Expand Down Expand Up @@ -200,7 +201,7 @@ CoreSync is the right choice when:
- Desktop applications caching data locally
- Field service apps that work without connectivity
- Multi-site databases that need bidirectional replication
- Any .NET application synchronizing between SQLite, SQL Server, and/or PostgreSQL
- Any .NET application synchronizing between SQLite, SQL Server, PostgreSQL, and/or MySQL

## When to Choose Debezium

Expand Down
2 changes: 1 addition & 1 deletion docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ title: CoreSync Documentation

# CoreSync

**CoreSync** is a .NET library for bidirectional data synchronization between databases. It supports SQLite, SQL Server, and PostgreSQL, letting you keep multiple database instances in sync — whether they're on the same machine or communicating over HTTP.
**CoreSync** is a .NET library for bidirectional data synchronization between databases. It supports SQLite, SQL Server, PostgreSQL, and MySQL, letting you keep multiple database instances in sync — whether they're on the same machine or communicating over HTTP.

## Pages

Expand Down
40 changes: 40 additions & 0 deletions src/CoreSync.MySql/CoreSync.MySql.csproj
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
<Project Sdk="Microsoft.NET.Sdk">

<PropertyGroup>
<Version Condition=" '$(APPVEYOR_BUILD_VERSION)' == '' ">0.0.1-local</Version>
<Version Condition=" '$(APPVEYOR_BUILD_VERSION)' != '' ">$(APPVEYOR_BUILD_VERSION)</Version>
<TargetFramework>netstandard2.0</TargetFramework>
<Authors>adospace</Authors>
<Description>CoreSync is a .NET standard library that provides data synchronization functions between databases. This is the MySQL provider assembly.</Description>
<Copyright>Adolfo Marinucci</Copyright>
<PackageProjectUrl>https://github.com/adospace/CoreSync</PackageProjectUrl>
<PackageLicenseExpression>MIT</PackageLicenseExpression>
<RepositoryUrl>https://github.com/adospace/CoreSync</RepositoryUrl>
<PackageTags>mysql mariadb data database sync synchronization .net</PackageTags>
<GeneratePackageOnBuild>true</GeneratePackageOnBuild>
<PackageRequireLicenseAcceptance>true</PackageRequireLicenseAcceptance>
<Nullable>enable</Nullable>
<LangVersion>latest</LangVersion>
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
<NoWarn>RS1036,NU1903,NU1902,NU1901,NU1904</NoWarn>
</PropertyGroup>

<PropertyGroup Condition="'$(Configuration)' == 'Release'">
<GenerateDocumentationFile>true</GenerateDocumentationFile>
<NoWarn>$(NoWarn);CS1591</NoWarn>
</PropertyGroup>

<ItemGroup>
<Compile Include="..\CoreSync\Validate.cs" Link="Validate.cs" />
</ItemGroup>

<ItemGroup>
<PackageReference Include="MySqlConnector" Version="2.6.1" />
<PackageReference Include="System.ComponentModel.Annotations" Version="5.0.0" />
</ItemGroup>

<ItemGroup>
<ProjectReference Include="..\CoreSync\CoreSync.csproj" />
</ItemGroup>

</Project>
26 changes: 26 additions & 0 deletions src/CoreSync.MySql/MySqlColumn.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
using System;

namespace CoreSync.MySql
{
internal class MySqlColumn
{
public MySqlColumn(string name, string dataType, string columnType, bool primaryKey = false)
{
Name = name;
DataType = dataType;
ColumnType = columnType;
IsPrimaryKey = primaryKey;
}

public string Name { get; }
public string DataType { get; }
public string ColumnType { get; }
public bool IsPrimaryKey { get; }

public bool IsGuidLike =>
(DataType.Equals("char", StringComparison.OrdinalIgnoreCase) ||
DataType.Equals("varchar", StringComparison.OrdinalIgnoreCase)) &&
(ColumnType.Equals("char(36)", StringComparison.OrdinalIgnoreCase) ||
ColumnType.Equals("varchar(36)", StringComparison.OrdinalIgnoreCase));
}
}
21 changes: 21 additions & 0 deletions src/CoreSync.MySql/MySqlCommandExtensions.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
using MySqlConnector;
using System.Threading;
using System.Threading.Tasks;

namespace CoreSync.MySql
{
internal static class MySqlCommandExtensions
{
public static async Task<long> ExecuteLongScalarAsync(this MySqlCommand command, CancellationToken cancellationToken = default)
{
var result = await command.ExecuteScalarAsync(cancellationToken);

if (result == null || result == System.DBNull.Value)
{
return 0;
}

return System.Convert.ToInt64(result);
}
}
}
9 changes: 9 additions & 0 deletions src/CoreSync.MySql/MySqlPrimaryColumnType.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
namespace CoreSync.MySql
{
internal enum MySqlPrimaryColumnType
{
Integer,
Text,
Blob
}
}
18 changes: 18 additions & 0 deletions src/CoreSync.MySql/MySqlSyncConfiguration.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
using JetBrains.Annotations;

namespace CoreSync.MySql
{
public class MySqlSyncConfiguration : SyncConfiguration
{
public string ConnectionString { get; }

internal MySqlSyncConfiguration([NotNull] string connectionString, [NotNull] MySqlSyncTable[] tables)
: base(tables)
{
Validate.NotNullOrEmptyOrWhiteSpace(connectionString, nameof(connectionString));
Validate.NotNullOrEmptyArray(tables, nameof(tables));

ConnectionString = connectionString;
}
}
}
118 changes: 118 additions & 0 deletions src/CoreSync.MySql/MySqlSyncConfigurationBuilder.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,118 @@
using JetBrains.Annotations;
using System;
using System.Collections.Generic;
using System.ComponentModel.DataAnnotations.Schema;
using System.Linq;

namespace CoreSync.MySql
{
/// <summary>
/// Provides a fluent API for building a <see cref="MySqlSyncConfiguration"/> that defines which MySQL tables
/// participate in synchronization and how they behave.
/// </summary>
public class MySqlSyncConfigurationBuilder
{
private readonly string _connectionString;
private readonly List<MySqlSyncTable> _tables = [];

public MySqlSyncConfigurationBuilder([NotNull] string connectionString)
{
Validate.NotNullOrEmptyOrWhiteSpace(connectionString, nameof(connectionString));
_connectionString = connectionString;
}

public MySqlSyncConfigurationBuilder Table(
[NotNull] string name,
Type? recordType = null,
SyncDirection syncDirection = SyncDirection.UploadAndDownload,
bool skipInitialSnapshot = false,
string? selectIncrementalQuery = null,
string? customSnapshotQuery = null)
{
Validate.NotNullOrEmptyOrWhiteSpace(name, nameof(name));

name = name.Trim();
if (_tables.Any(_ => string.CompareOrdinal(_.Name, name) == 0))
{
throw new InvalidOperationException($"Table with name '{name}' already added");
}

_tables.Add(new MySqlSyncTable(name, recordType, syncDirection, skipInitialSnapshot, selectIncrementalQuery, customSnapshotQuery));
return this;
}

public MySqlSyncConfigurationBuilder Table<T>(
string? name = null,
SyncDirection syncDirection = SyncDirection.UploadAndDownload,
bool skipInitialSnapshot = false,
string? selectIncrementalQuery = null,
string? customSnapshotQuery = null)
{
if (name == null)
{
var tableAttribute = (TableAttribute?)Attribute.GetCustomAttribute(typeof(T), typeof(TableAttribute));
if (tableAttribute != null)
{
name = tableAttribute.Name;
}
}

name ??= typeof(T).Name;

if (_tables.Any(_ => string.CompareOrdinal(_.Name, name) == 0))
{
throw new InvalidOperationException($"Table with name '{name}' already added");
}

return Table(name, typeof(T), syncDirection, skipInitialSnapshot, selectIncrementalQuery, customSnapshotQuery);
}

public MySqlSyncConfigurationBuilder SkipColumns(params string[] columnNames)
{
var lastTable = _tables.LastOrDefault()
?? throw new InvalidOperationException("SkipColumns requires a table");

lastTable.SkipColumns = columnNames ?? throw new ArgumentNullException(nameof(columnNames));
return this;
}

public MySqlSyncConfigurationBuilder SkipColumnsOnInsertOrUpdate(params string[] columnNames)
{
var lastTable = _tables.LastOrDefault()
?? throw new InvalidOperationException("SkipColumnsOnInsertOrUpdate requires a table");

lastTable.SkipColumnsOnInsertOrUpdate = columnNames ?? throw new ArgumentNullException(nameof(columnNames));
return this;
}

public MySqlSyncConfigurationBuilder SelectIncrementalQuery(string selectIncrementalQuery)
{
if (string.IsNullOrWhiteSpace(selectIncrementalQuery))
{
throw new ArgumentException($"'{nameof(selectIncrementalQuery)}' cannot be null or whitespace", nameof(selectIncrementalQuery));
}

var lastTable = _tables.LastOrDefault()
?? throw new InvalidOperationException("SelectIncrementalQuery requires a table");

lastTable.SelectIncrementalQuery = selectIncrementalQuery;
return this;
}

public MySqlSyncConfigurationBuilder CustomSnapshotQuery(string customSnapshotQuery)
{
if (string.IsNullOrWhiteSpace(customSnapshotQuery))
{
throw new ArgumentException($"'{nameof(customSnapshotQuery)}' cannot be null or whitespace", nameof(customSnapshotQuery));
}

var lastTable = _tables.LastOrDefault()
?? throw new InvalidOperationException("CustomSnapshotQuery requires a table");

lastTable.CustomSnapshotQuery = customSnapshotQuery;
return this;
}

public MySqlSyncConfiguration Build() => new MySqlSyncConfiguration(_connectionString, _tables.ToArray());
}
}
Loading