Coder Social home page Coder Social logo

gatling-jdbc's Introduction

Gatling JDBC Extension

JDBC support for Gatling

Build Status Maven Central

The JDBC extension for Gatling was originally created to accompany a blog post that shows how to extend Gatling. Currently, five SQL operations are being supported. See below for the usage.

❗❗ Even more Attention ❗❗

The development of this project continues in this fork. Please refrain from cloning/forking this repository or creating issue. Please use the other one!

❗ Attention ❗

In order to avoid conflicts with io.gatling:gatling-jdbc the artifact name has been changed with version 2.0.1. Instead of gatling-jdbc it is now called jdbc-gatling (see issue #8). Apart from this, nothing changes. All package names etc. stayed the same.

Usage

libraryDependencies += "de.codecentric" %% "gatling-jdbc" % "version"

General

In order to use the JDBC functionality, your simulation has to import de.codecentric.gatling.jdbc.Predef._. The JDBC configuration is done via jdbc, e.g.:

val jdbcConfig = jdbc
  .url("jdbc:h2:mem:test;DB_CLOSE_ON_EXIT=FALSE")
  .username("sa")
  .password("sa")
  .driver("org.h2.Driver")

Those are currently all the options that can be provided and that have to be provided.

The entry point for the operations is the jdbc() method. The method itself takes a request name as parameter. This name will appear in the reports to represent the operation that follows.

CREATE TABLE

Creating a table is done via jdbc().create(). In order to ease the creation of arbitrarily many columns, the helper class import de.codecentric.gatling.jdbc.builder.column.ColumnHelper._ was created. It is recommended to use it. The datatype of every column has to be provided as a string. Additionally, constraints are optional, but also have to be passed as strings. E.g.:

scenario("createTable").
    exec(jdbc("bar table")
      .create()
      .table("bar")
      .columns(
        column(
          name("abc"),
          dataType("INTEGER"),
          constraint("PRIMARY KEY")
        ),
        column(
          name("ac"),
          dataType("INTEGER")
        )
      )
    )

INSERT

Insertion is done via jdbc().insert(). For where to insert, two options are possible. Suppose you have the table from the example above. You can insert the values either by relying on the indices:

exec(jdbc("insertion")
  .insert()
  .into("bar")
  .values("${n}, ${n}")
)

or by using the column names:

exec(jdbc("insertion")
  .insert()
  .into("bar (abc, ac)")
  .values("${n}, ${n}")
)

SELECT

In contrast to the previous operations, select directly requires a parameter and is called via jdbc().select(<what>). The intention is to closely resemble the SQL syntax. Using where() is optional for SELECT. Therefore, the following two ways are both valid:

exec(jdbc("selection")
  .select("*")
  .from("bar")
)

and

exec(jdbc("selection")
  .select("*")
  .from("bar")
  .where("abc=4")
)

Of course, as parameter to select(), every column or combination of columns can be used, as with basic SQL.

DELETE

Deletion starts from jdbc().delete(). Here, the where clause is optional again. In order to delete certain values the following works:

repeat(5, "n"){
    exec(jdbc("deletion")
        .delete()
        .from("bar")
        .where("abc=${n}")  
    )
}

Alternatively, in order to delete everything:

exec(jdbc("deletion")
    .delete()
    .from("bar")
)

Please be careful, since no additional validation is being performed and you might lose some data.

DROP TABLE

The last operation that is being supported is DROP TABLE via jdbc().drop() The method only takes a single parameter, the table name. Please be careful again, which table you drop. Dropping the "bar" table from the first example can be done in the following way:

jdbc("drop bar table").drop().table("bar")

Checks

Currently, checks are only implemented for SELECT. When importing de.codecentric.gatling.jdbc.Predef._ two types of checks are provided. The first type is the SimpleCheck.

SimpleCheck

The simpleCheck method (importet via Predef) allows for very basic checks. This method takes a function from List[Map[String, Any]] to Boolean. Each element in the list represents a row and the map the individual columns. Checks are simply appended to the selection, e.g.:

exec(jdbc("selection")
  .select("*")
  .from("bar")
  .where("abc=4")
  .check(simpleCheck(result => result.head("FOO") == 4))
)

A SELECT without a WHERE clause can also be validated with a simpleCheck.

There is also another type of check that is more closely integrated with Gatling, the CheckBuilders.

CheckBuilder

CheckBuilder is actually a class provided by Gatling. Based on the Gatling classes, Gatling JDBC provides two types of them. The JdbcAnyCheckBuilder object contains the instances SingleAnyResult and ManyAnyResults. Both can be used in the tests quickly by calling either jdbcSingleResponse or jdbcManyResponse.

The difference between the two is that the single response extracts the head out of the list of results. So you can only verify a Map[String, Any]. Whereas the many response, like the simple checks, returns a List[Map[String, Any]]. Validation is performed via the Gatling API. E.g. checking a single result can look like this:

exec(jdbc("selectionSingleCheck")
  .select("*")
  .from("bar")
  .where("abc=4")
  .check(jdbcSingleResponse.is(Map[String, Any]("ABC" -> 4, "FOO" -> 4)))
)

This validates the data in the two columns "ABC" and "FOO". Please note explicit typing of the map. Without it the compiler will complain.

A check with multiple results doesn't look very different:

exec(jdbc("selectionManyCheck")
  .select("*")
  .from("bar")
  .where("abc=4 OR abc=5")
  .check(jdbcManyResponse.is(List(
    Map("ABC" -> 4, "FOO" -> 4),
    Map("ABC" -> 5, "FOO" -> 5)))
  )
)

The advantage those CheckBuilder provide is that they can access certain functionality provided by the Gatling interfaces and classes they extend. The most important one is the possibility to save the result of a selection to the current session. By calling saveAs after a check you can place the result in the session under the given name. So e.g. if you want to store the result of the single check you can do it like this:

exec(jdbc("selectionSingleCheckSaving")
  .select("*")
  .from("bar")
  .where("abc=4")
  .check(jdbcSingleResponse.is(Map[String, Any]("ABC" -> 4, "FOO" -> 4))
  .saveAs("myResult"))
)

Final

Covering all SQL operations is a lot of work and some special commands might not be required for performance tests. Please keep in mind that the state of this Gatling extension can be considered experimental. Feel free to leave comments and create pull requests.

Publishing

Firstly, you gotta have in your home .sbt/1.0/sonatype.sbt configured to contain your username and password for Sonatype. Secondly, open the sbt shell an perform the following steps:

  1. set pgpSecretRing := file("/home/<user>/.sbt/gpg/secring.asc") or where ever it is
  2. release

Executing the intergration tests

If you have to run Docker on your machine as sudo, then to execute the integration tests, sbt has to be started as sudo, too. Only sudo sbt gatling:test will then be allowed to start the container of the databases.

gatling-jdbc's People

Contributors

rbraeunlich avatar riggs333 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

gatling-jdbc's Issues

Multiple jdbc connections

Hi @rbraeunlich, I have to execute SQL queries from different DBs during scenario. Tell me please, is it possible to create multiple jdbc connections?
Like first:

val jdbcConfig1 = jdbc
  .url("jdbc:h2:db:1")
  .username("sa")
  .password("sa")
  .driver("org.h2.Driver")

And second:

val jdbcConfig2= jdbc
  .url("jdbc:postgresql://db:another")
  .username("sa")
  .password("sa")
  .driver("org.postgresql.Driver")

If it possible, how can I specify database to make request?

exec(
  jdbc("selection").db("jdbcConfig2") // ???
  .select("*")
  .from("bar")
  .where("abc=4")
)

How can I connect to a db after my scenario is run?

Hi there 👋 I'm trying to connect to my organisation's db and perform a query after my scenario is run. Unfortunately I can't past my code here but essentially this is what I have:

  • my scenario
  • and an after hook that, naturally, is run after the completion of the scenario.
    I want to connect to the db inside this after hook, here's an example below

kmkm

However I always get this error message

kjdskjd

In my gradle build script I have this dependency `io.quarkus:quarkus-jdbc-postgresql`, so I should have the driver. This leads me to conclude that quarkus doesn run to execute gatling (which makes sense) and therefore RIP driver :( do u know a way to do this? Thanks nonetheless

java.lang.NoClassDefFoundError: while running on latest build

java.lang.NoClassDefFoundError: scala/collection/compat/Factory$
at scalikejdbc.DBSession.list(DBSession.scala:295)
at scalikejdbc.DBSession.list$(DBSession.scala:294)
at scalikejdbc.ActiveSession.list(DBSession.scala:837)
at scalikejdbc.DBSessionWrapper.$anonfun$list$1(DBSessionWrapper.scala:52)
at scalikejdbc.DBSessionWrapper.$anonfun$withAttributesSwitchedDBSession$1(DBSessionWrapper.scala:34)
at scalikejdbc.DBSessionAttributesSwitcher.withSwitchedDBSession(DBSessionAttributesSwitcher.scala:31)
at scalikejdbc.DBSessionWrapper.withAttributesSwitchedDBSession(DBSessionWrapper.scala:33)
at scalikejdbc.DBSessionWrapper.list(DBSessionWrapper.scala:52)
at scalikejdbc.SQLToList.result(SQL.scala:931)
at scalikejdbc.SQLToList.result$(SQL.scala:930)
at scalikejdbc.SQLToListImpl.result(SQL.scala:944)
at scalikejdbc.SQLToListImpl.result(SQL.scala:944)
at scalikejdbc.SQLToResult.$anonfun$apply$21(SQL.scala:811)
at scalikejdbc.SQLToResult.apply(SQL.scala:817)
at scalikejdbc.SQLToResult.apply$(SQL.scala:805)
at scalikejdbc.SQLToListImpl.apply(SQL.scala:944)
at de.codecentric.gatling.jdbc.action.JdbcSelectAction.$anonfun$execute$3(JdbcSelectAction.scala:45)
at scalikejdbc.DBConnection.autoCommit(DBConnection.scala:242)
at scalikejdbc.DBConnection.autoCommit$(DBConnection.scala:240)
at scalikejdbc.DB.autoCommit(DB.scala:60)
at scalikejdbc.DB$.$anonfun$autoCommit$1(DB.scala:216)
at scalikejdbc.LoanPattern.using(LoanPattern.scala:18)
at scalikejdbc.LoanPattern.using$(LoanPattern.scala:16)
at scalikejdbc.DB$.using(DB.scala:139)
at scalikejdbc.DB$.autoCommit(DB.scala:215)
at de.codecentric.gatling.jdbc.action.JdbcSelectAction.$anonfun$execute$2(JdbcSelectAction.scala:44)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:658)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
at java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1402)
at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)

Queries running sequentially and not parallel for virtual users

Hi,
I am trying to use jdbc protocol with 10 virtual users using atOnce(10) where I expected the queries in my data set would run in parallel. But the queries are running sequentially where one user starts and complete the query and then the next query is picked by next user.

Connection Pool settings

Currently we are using default connection pool configuration,

ConnectionPool.singleton(url, username, pwd)

it would be nice to be able to configure the scalikejdbc connection pool
Seems that in some cases max size 8 is not enough when doing load testing.

Unable to create more than 8 connections to MySQL database

Hi,

I am trying to send through enough load through to our RDS mysql database using jdbc-gatling but I am unable to get to more than 4 connections from a Macbook pro and not more than 8 connections from a Windows 7 machine.
Regardless of the number of concurrent users I spawn these are all the connections I can get to

Here below I set the connectionpoolsettings to go upto 200. But it has not helped

val settings = ConnectionPoolSettings(initialSize = 100, maxSize = 200)
 val jdbcConfig = jdbc.url("jdbc:mysql://blah-blah-blah")
    .username(dbusername)  // TODO Change this to perpetual or on demand Authorization
    .password(dbpassword)
    .driver("com.mysql.cj.jdbc.Driver")
    .connectionPoolSettings(settings)
    .build

I am using the following versions of the various dependencies I am using

     <gatling.version>3.3.1</gatling.version>
      <scala-maven-plugin.version>3.3.1</scala-maven-plugin.version>
     <gatling-maven-plugin.version>3.0.5</gatling-maven-plugin.version>
      <jdbc-gatling.version>2.3.0</jdbc-gatling.version>
      <mysql-connector.version>8.0.15</mysql-connector.version>

Rename artifact so it doesn't clash with `io.gatling:gatling-jdbc`

Hi there,

I know this request is a PITA, all the more as this project is actively developed and is seems to be gaining more traction.
Sadly, the artifact id you picked clashes with io.gatling:gatling-jdbc (that contains the code for jdbcFeeder).
It brings confusion and one can't have both when using Gatling's bundle distrib.

Maybe at some point we'll forbid third parties to use the gatling- prefix to avoid such confusion and clashes, just like maven does.

Could you please consider renaming it?

Kind regards

Is gatling-jdbc compatible with Gatling 3.0?

Hi,

I'm a bit a of a newb to Gatling / Scala - but I do have my scenario working such that it can make http() calls, but my gatling-jdbc jdbc() call fails.

As per the example code below, using the sqlSelect scenario, I get an UnsupportedOperationException.
The httpGet scenario works as expected.

Code:

import io.gatling.http.Predef._
import io.gatling.core.Predef._
import de.codecentric.gatling.jdbc.Predef._

class DatabaseSimulation extends Simulation {

  private val httpConf = http
  private val jdbcConfig = jdbc
    .url("jdbc:postgresql://...")
    .username("postgres")
    .password("password")
    .driver("org.postgresql.Driver")

  private val httpGet =
    scenario("getF5VersionScenario").exec(
      http("getF5Version").
        get("https://...") )

  private val sqlSelect =
    scenario("f5JdbcServerVersionScenario").exec(
      jdbc("selectVersion").select("*").from("dual")
    )

  setUp(
//    httpGet.inject(atOnceUsers(1))
    sqlSelect.inject(atOnceUsers(1))
  ).
    protocols(jdbcConfig).
    protocols(httpConf)

}

Exception:

java.lang.UnsupportedOperationException: empty.reduceLeft
	at scala.collection.LinearSeqOptimized.reduceLeft(LinearSeqOptimized.scala:135)
	at scala.collection.LinearSeqOptimized.reduceLeft$(LinearSeqOptimized.scala:134)
	at scala.collection.immutable.List.reduceLeft(List.scala:85)
	at scala.collection.TraversableOnce.reduce(TraversableOnce.scala:207)
	at scala.collection.TraversableOnce.reduce$(TraversableOnce.scala:207)
	at scala.collection.AbstractTraversable.reduce(Traversable.scala:104)
	at io.gatling.core.protocol.ProtocolComponentsRegistry.onStart(Protocol.scala:76)
	at io.gatling.core.structure.PopulationBuilder.build(ScenarioBuilder.scala:102)
	at io.gatling.core.scenario.SimulationParams.$anonfun$scenarios$1(Simulation.scala:194)
	at scala.collection.immutable.List.map(List.scala:282)
	at io.gatling.core.scenario.SimulationParams.scenarios(Simulation.scala:194)
	at io.gatling.app.Runner.run0(Runner.scala:92)
	at io.gatling.app.Runner.run(Runner.scala:61)
	at io.gatling.app.Gatling$.start(Gatling.scala:74)
	at io.gatling.app.Gatling$.fromMap(Gatling.scala:42)
	at myorg.sim.OneOffSysProp$.delayedEndpoint$myorg$sim$OneOffSysProp$1(OneOffSysProp.scala:27)
	at myorg.sim.OneOffSysProp$delayedInit$body.apply(OneOffSysProp.scala:11)
	at scala.Function0.apply$mcV$sp(Function0.scala:34)
	at scala.Function0.apply$mcV$sp$(Function0.scala:34)
	at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:12)
	at scala.App.$anonfun$main$1$adapted(App.scala:76)
	at scala.collection.immutable.List.foreach(List.scala:388)
	at scala.App.main(App.scala:76)
	at scala.App.main$(App.scala:74)
	at myorg.sim.OneOffSysProp$.main(OneOffSysProp.scala:11)
	at myorg.sim.OneOffSysProp.main(OneOffSysProp.scala)
Exception in thread "main" java.lang.UnsupportedOperationException: empty.reduceLeft
	at scala.collection.LinearSeqOptimized.reduceLeft(LinearSeqOptimized.scala:135)
	at scala.collection.LinearSeqOptimized.reduceLeft$(LinearSeqOptimized.scala:134)
	at scala.collection.immutable.List.reduceLeft(List.scala:85)
	at scala.collection.TraversableOnce.reduce(TraversableOnce.scala:207)
	at scala.collection.TraversableOnce.reduce$(TraversableOnce.scala:207)
	at scala.collection.AbstractTraversable.reduce(Traversable.scala:104)
	at io.gatling.core.protocol.ProtocolComponentsRegistry.onStart(Protocol.scala:76)
	at io.gatling.core.structure.PopulationBuilder.build(ScenarioBuilder.scala:102)
	at io.gatling.core.scenario.SimulationParams.$anonfun$scenarios$1(Simulation.scala:194)
	at scala.collection.immutable.List.map(List.scala:282)
	at io.gatling.core.scenario.SimulationParams.scenarios(Simulation.scala:194)
	at io.gatling.app.Runner.run0(Runner.scala:92)
	at io.gatling.app.Runner.run(Runner.scala:61)
	at io.gatling.app.Gatling$.start(Gatling.scala:74)
	at io.gatling.app.Gatling$.fromMap(Gatling.scala:42)
	at myorg.sim.OneOffSysProp$.delayedEndpoint$myorg$sim$OneOffSysProp$1(OneOffSysProp.scala:27)
	at myorg.sim.OneOffSysProp$delayedInit$body.apply(OneOffSysProp.scala:11)
	at scala.Function0.apply$mcV$sp(Function0.scala:34)
	at scala.Function0.apply$mcV$sp$(Function0.scala:34)
	at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:12)
	at scala.App.$anonfun$main$1$adapted(App.scala:76)
	at scala.collection.immutable.List.foreach(List.scala:388)
	at scala.App.main(App.scala:76)
	at scala.App.main$(App.scala:74)
	at myorg.sim.OneOffSysProp$.main(OneOffSysProp.scala:11)
	at myorg.sim.OneOffSysProp.main(OneOffSysProp.scala)

Process finished with exit code 1

I'm using a Gatling 3.0.0-RC4 "maven archetype" project.

Using gatling-jdbc by adding this to the pom.xml:

    <dependency>
      <groupId>de.codecentric</groupId>
      <artifactId>gatling-jdbc_2.12</artifactId>
      <version>1.0.1</version>
    </dependency>

As per the title, should gatling-jdbc work with the new version of gatling-jdbc? Or am I doing something else wrong?

Add possibility to type result of select

ScalikeJDBC offers a way to directly map results into a certain type by having the companion objects implement SQLSyntaxSupport[Member]. The JdbcSelectAction(Creator) should offer a way to make use of this, instead of always having List[Map[String, Any]].
There should also be a JdbcCheckBuilder added for this.

How to execute SQL query during scenario correct ?

How to execute usual SQL query to get object ID and put this ID into a method like a parametr?
I have write the code for SQL selection
val technic_root_id = exec(jdbc("selection").select("technic_root_id").from("ods_req_car.technic_request").where("id = ${car_request_id}"))
In scenarion I call it through this exec:
.exec(Technic.viewTechnicCard(Technic.technic_root_id.toString))

In output log I see the following
GET http://server.host/api/tech/ChainBuilder(List(JdbcSelectionWithWhereActionBuilder(%3Cfunction1%3E,%3Cfunction1%3E,%3Cfunction1%3E,io.gatling.core.session.el.ElCompiler$$$Lambda$338/879423823@6464ef69)))?loadTree=true

Seems like this is not what I want. This is not ID of my object after tech/ part of URL.
How can I execute SQL query during scenario and put the result of query into method ?

Can't make a request

Hi, Ronny.

I've installed your extension with maven and now trying to use it to connect to oracle db. My code is like the following:

  val jdbcConfig = jdbc
    .url("jdbc:oracle:thin:@//dbhost:1521/scheme")
    .username("user")
    .password("pass")
    .driver("oracle.jdbc.OracleDriver")
val request=
  exec(jdbc("selection")
    .select("age")
    .from("people"))

In .pom file I have:

	          <dependency>
			<groupId>com.oracle</groupId>
			<artifactId>ojdbc8</artifactId>
			<version>12.2.0.1</version>
		</dependency>
		<dependency>
			<groupId>de.codecentric</groupId>
			<artifactId>gatling-jdbc_2.12</artifactId>
			<version>1.0.0</version>
		</dependency>

My scenario consists of only one action. When I launch my script, in a log I can see only this:

================================================================================
2018-05-30 15:30:44                                           0s elapsed
---- Requests ------------------------------------------------------------------
> Global                                                   (OK=0      KO=1     )
> selection                                                (OK=0      KO=1     )
---- Errors --------------------------------------------------------------------
> <no-message>                                                        1 (100,0%)

---- SC01 ---------------------------------------------------------------------
[##########################################################################]100%
          waiting: 0      / active: 0      / done:1     
================================================================================

Can you advice me please, what I'm doing wrong?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.