wanlitao / hangfireextension

Hangfire Extension plugins

hangfireextension's Issues

How Can we configure Auto Clean up for sqlitedb after specific time?

I am use Hangfire.Core Version=1.7.8.0, Hangfire.SQLite Version=1.4.2.0

Records in JobParameter and State table are not deleted when expired

Hi Wanlitao,

I expected that records in JobParameter and State is to be deleted when expired job deleted becase there is a foreign key to job table, But, it didn't.

Any idea why?

Regards
Joe

Maintenance needed

Are you planning to release a new version with the current issues addressed?

If not, may I suggest that you invite new maintainers to help you in this task? I'd be interested to help.

If a Job fails 3 times in a row, it will never run to completion

If a Job fails, we try to set a RetryCount JobParameter. At some point this overwrites all the other JobParameters (e.g. CurrentCulture and CurrentUICulture) because of the bug in SQLiteStorageConnection.SetJobParameter. Therefore, we get IvalidOperationException in SQLiteStorageConnection.GetJobParameter for RetryCount.

Problem with connection string without semicolon

Using the connection string Data Source=data.db, an exception was throwned with the error Could not find connection string with name 'Data Source=data.db' in application config file. This error not happen if connection string is Data Source=data.db;

There is any configuration way to decrease HangFire sqlitedb file after specific size or specific time

HangFire sqlitedb file get bigger every day
I want any configuration way to delete records from all tables that created from one week ago

Sqlite storage makes asp.net core application very slow

I'm trying to use Hangfire with Sqlite, and when I do that my application gets very slow.

Using Visual Studio 2017, I created a Asp.Net Core Web API project.

I added the configuration to Startup.cs:

        public void ConfigureServices(IServiceCollection services)
        {
            // Add framework services.
            services.AddMvc();
            //services.AddHangfire(x => x.UseRedisStorage("10.0.3.109"));
            //services.AddHangfire(x => x.UseSqlServerStorage("Data Source=(localdb)\\mssqllocaldb;Database=aspnet-TesteUsuario-b590c41d-8723-4abc-abe0-1c93ac2af944"));

            services.AddHangfire(x => x.UseSQLiteStorage("Data Source=c:/temp/hangfire.db;Version=3;"));
            services.AddSingleton<IBackgroundService, BackgroundService>();
            services.AddSingleton<BackgroundService, BackgroundService>();
        }

        // This method gets called by the runtime. Use this method to configure the HTTP request pipeline.
        public void Configure(IApplicationBuilder app, IHostingEnvironment env, ILoggerFactory loggerFactory)
        {
            loggerFactory.AddConsole(Configuration.GetSection("Logging"));
            loggerFactory.AddDebug();

            app.UseMvc();
            var option = new BackgroundJobServerOptions { WorkerCount = 1 };
            app.UseHangfireServer(option);
            app.UseHangfireDashboard();             
        }

I then added a Hangfire call to my Get methd on the values controller

        // GET api/values
        [HttpGet]
        public IEnumerable<string> Get()
        {
            BackgroundJob.Enqueue(() => _service.Process(1));
            return new string[] { "value1", "value2" };
        }

When I use Redis the controller responds in around 7-15 ms.
Executed action TesteHangfire.Controllers.ValuesController.Get (TesteHangfire) in 7.6838ms
When I use SqlServer the controller responds in around 70-150 ms.
Executed action TesteHangfire.Controllers.ValuesController.Get (TesteHangfire) in 78.5856ms
But when I use Sqlite the controller takes more than 28000 ms
Executed action TesteHangfire.Controllers.ValuesController.Get (TesteHangfire) in 28842.5443ms

At first I was getting all kind of erros when writing to DB, but then I added this config and the errors stopped, but the application is still slow:

            var option = new BackgroundJobServerOptions { WorkerCount = 1 };
            app.UseHangfireServer(option);

Encrypted support in .Net Standard 2.0 with .Net Framework 4.8

I am using sqlite in .Net standard 2.0 with .Net framework 4.8 so I added all dll through nuget.

string connectionString = "Data Source="C:\Program Data\sampleproject.db;Password=p123".

In first case:
System.Data.Sqlite,System.Data.sqlite.core, system.data.sqlite.linq. While passing datasource path with password. It gives exception while opening a Sqliteconnection:
"SQL logic error
Cannot use "Password" connection string property: library was not built with encryption support, please see "https://www.sqlite.org/see" for more information".
so I added sqlitepcl.raw, sqlitepclraw.core. No change in result I got same exception.

In second scenario, in fresh way to add dll. I added dll :
sqlite-net-base, sqlite-net-sqlcipher, sqlitepclraw.bundle.e-sqlite3, sqlitepclraw.core. In this case, I tried to call connection with Sqliteasyncconnection and sqliteconnection. Both times exception says:
"You need to call SQLitePCL.raw.SetProvider(). If you are using a bundle package, this is done by calling SQLitePCL.Batteries.Init()".

In third sceanrio, I used Microsoft.data.sqlite.core and sqlitepclraw.bundle_e_sqlcipher according to
"https://docs.microsoft.com/en-us/dotnet/standard/data/sqlite/encryption?tabs=netcore-cli". I am getting exception while creating connection.
"The process has no package identity. (Exception from HRESULT: 0x80073D54)".

Which scenario will suit to allow me password encrypted db file to create and connect. What modification required. Thanks in advance.

support .net core?

Hello
Can you Port this Extension for .Net Core?
support .NetStandard 1.6

Database is locked error

When using SQLite with a recurring job the error SQLite error (5): database is locked is shown in the output window many times, after that rows are inserted into the State table and they stay with the Processing state.

I'm using Hangfire in a Windows Service following the docs.

You need to call SQLitePCL.raw.SetProvider()

running on .net core 2.0 webApi project

You need to call SQLitePCL.raw.SetProvider(). If you are using a bundle package, this is done by calling SQLitePCL.Batteries.Init().

connection string = Data Source=:memory:;

at startup db do not exist

Update Install.sql to alter tables instead of creating them on every run.

Hi,
Every time running .UseSQLiteStorage(ConnectionString, options) it attempts to create Hangfire-related tables even if they already exist. This leads to sql-exception and failure.

Lock timeout exceptions happen every time

My workers crash every time:

Error occurred during execution of 'Worker #2d5f9ca3' process. Execution will be retried (attempt 1 of 2147483647) in 00:00:01 seconds.

System.ApplicationException: This operation returned because the timeout period expired. (Exception from HRESULT: 0x800705B4)
   at System.Threading.ReaderWriterLock.AcquireWriterLockInternal(Int32 millisecondsTimeout)
   at System.Threading.ReaderWriterLock.AcquireWriterLock(TimeSpan timeout)
   at Hangfire.SQLite.SQLiteStorage.CreateAndOpenConnection(Boolean isWriteLock)
   at Hangfire.SQLite.SQLiteStorage.UseConnection[T](Func`2 func, Boolean isWriteLock)
   at Hangfire.SQLite.SQLiteStorage.UseConnection(Action`1 action, Boolean isWriteLock)
   at Hangfire.SQLite.SQLiteFetchedJob.RemoveFromQueue()
   at Hangfire.Server.Worker.Execute(BackgroundProcessContext context)
   at Hangfire.Server.ServerProcessExtensions.Execute(IServerProcess process, BackgroundProcessContext context)
   at Hangfire.Server.AutomaticRetryProcess.Execute(BackgroundProcessContext context)

   at System.Threading.ReaderWriterLock.AcquireWriterLockInternal(Int32 millisecondsTimeout)
   at System.Threading.ReaderWriterLock.AcquireWriterLock(TimeSpan timeout)
   at Hangfire.SQLite.SQLiteStorage.CreateAndOpenConnection(Boolean isWriteLock) in C:\Work\External\HangfireExtension\Hangfire.SQLite\SQLiteStorage.cs:line 214
   at Hangfire.SQLite.SQLiteStorage.UseConnection[T](Func`2 func, Boolean isWriteLock) in C:\Work\External\HangfireExtension\Hangfire.SQLite\SQLiteStorage.cs:line 176
   at Hangfire.SQLite.SQLiteStorage.UseConnection(Action`1 action, Boolean isWriteLock) in C:\Work\External\HangfireExtension\Hangfire.SQLite\SQLiteStorage.cs:line 163
   at Hangfire.SQLite.SQLiteJobQueue.Dequeue(String[] queues, CancellationToken cancellationToken) in C:\Work\External\HangfireExtension\Hangfire.SQLite\SQLiteJobQueue.cs:line 69
   at Hangfire.SQLite.SQLiteStorageConnection.FetchNextJob(String[] queues, CancellationToken cancellationToken) in C:\Work\External\HangfireExtension\Hangfire.SQLite\SQLiteStorageConnection.cs:line 68
   at Hangfire.Server.Worker.Execute(BackgroundProcessContext context)
   at Hangfire.Server.ServerProcessExtensions.Execute(IServerProcess process, BackgroundProcessContext context)
   at Hangfire.Server.AutomaticRetryProcess.Execute(BackgroundProcessContext context)

Not sure if it happens due to my code (I integrated Hangfire to ASP.NET Core, but used Hangfire.SQLite without changes).

Duplicate key error when navigate to job details

on some random jobs I get this error when I go to job details page :

An unhandled exception was thrown by the application. An item with the same key has already been added. Key: RetryCount

I get a blank page and I see that in my application logs.

It seems similar to this issue : gottscj/Hangfire.Mongo#70 ?

and when I have a look to JobParameter table I see same job id with multiple RetryCount.

能不能把 dapper 的依赖去掉呢 仅仅依赖sqlite自身的库

项目里用其他的orm 了从而产生了冲突呢。。我跟你同个city哈

ConnectionString detection is not very robust

I am trying to pass a connectionString as a parameter in the method GlobalConfiguration.Configuration.UseSQLiteStorage. But it tries to treat the connectionstring as connectionstring name and fails to find it in configuration.

Looking at the code
https://github.com/wanlitao/HangfireExtension/blob/master/src/Hangfire.SQLite/SQLiteStorage.cs#L308
it seems like it is trying to find ";" in the string. I had to explicitly add it at the end.

Hangfire Dashboard: Cannot implicitly convert type ‘long’ to ‘int?’. An explicit conversion exists (are you missing a cast?)

Hi
We are using Hangfire 1.6.16 with SQLite 1.4.1 backend.
Whenever we click in the Dashboard on the Jobs-Tab to see the jobs lis the following exception is show:
Cannot implicitly convert type 'long' to 'int?'. An explicit conversion exists (are you missing a cast?)

StackTrace:
[RuntimeBinderException: Cannot implicitly convert type 'long' to 'int?'. An explicit conversion exists (are you missing a cast?)]
CallSite.Target(Closure , CallSite , Object ) +206
Hangfire.SQLite.<>c__DisplayClass9_0.b__0(DbConnection connection) +534
Hangfire.SQLite.SQLiteStorage.UseConnection(Func2 func, Boolean isWriteLock) +62
Hangfire.SQLite.SQLiteJobQueueMonitoringApi.GetEnqueuedAndFetchedCount(String queue) +270
Hangfire.SQLite.SQLiteMonitoringApi.Queues() +701
Hangfire.Dashboard.Pages.QueuesPage.Execute() +221
Hangfire.Dashboard.RazorPage.TransformText(String body) +31
Hangfire.Dashboard.RazorPageDispatcher.Dispatch(DashboardContext context) +86
Hangfire.Dashboard.<>c__DisplayClass1_2.b__1(IDictionary2 env) +466
Microsoft.Owin.Mapping.d__0.MoveNext() +461
System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() +31
System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) +62
Microsoft.Owin.Host.SystemWeb.IntegratedPipeline.d__5.MoveNext() +203
System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() +31
System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) +62
Microsoft.Owin.Host.SystemWeb.IntegratedPipeline.d__2.MoveNext() +193
System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() +31
Microsoft.Owin.Host.SystemWeb.IntegratedPipeline.StageAsyncResult.End(IAsyncResult ar) +119
System.Web.AsyncEventExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute() +365
System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously) +134

Can anybode help on this issue?
Thank you in advance
Adrian

Work Item is picked and processed multiple times.

Looks like there is some synchronization and/or DB locking issue.
Same Work Item processed multiple times (even when max attempts count set) and then stuck in processing forever. In case of large number of workers it might lead to the situation when same work item is being processed by multiple workers in the same time.
Any idea or similar issues?

Updated NuGet Publication

Hi, are there any plans to push @agrocks "IF NOT EXISTS" changes out to NuGet? Just trying to decide whether to build locally or wait for a NuGet update. Thank you!

It should be documented that the connection string must include DateTimeFormat=Ticks

I was seeing jobs executed in the future! This is because of SQLite provider mapping of DateTime fields which sets the Kind property to Unspecified, leading to bad computations by Hangfire. Adding ;DateTimeFormat=Ticks to the connection string solved the issue.

This is also somewhat related to #18 as you have now a good excuse to use a ; in the connection string. 😉

Unable to find any use example.

Can you please provide a "How To Use" example in your ReadMe?

Make given string is valid connection string by the property like "data source" or using System.Data.Common.DbConnectionStringBuilder class instead of checking the string has ';'.

HangfireExtension/src/Hangfire.SQLite/SQLiteStorage.cs

Line 308 in e692a4d

return nameOrConnectionString.Contains(";");

Hi Wu, about to update hangfire extension

你好，我最近在想用hangfire的sqlite插件，找了半天网上就找到你这个extension，貌似是你自己扩展的，但是是在.net framework 4.5的版本上扩展的，然后用的System.Data.SQLite, System.Data.SQLite.EF 和 System.Data.SQLite.Linq都是基于这个版本的，我们项目现在用的是.net core 2.2的版本，我想弄个基于.net core 2.2的hangfire sqlite版本，想找你帮我看下你之前扩展的这个是在哪里加的sqlite。

万分谢谢，能不能加下我wechat: ljp12008