Thursday, November 17, 2022

Backing Up MySQL Part 2: Percona XtraBackup

There’s no doubt about it – if you’ve ever heard of Percona, you’ve heard of XtraBackup as well. XtraBackup is one of the primary Percona’s offerings in the backup space: the tool is famous amongst DBAs as one of the primary open-source utilities to take hot backups. The tool is known to avoid locking databases during its backup procedures – of course, it has a couple of limitations unique to itself, but that’s not an obstacle for experienced database administrators across the globe: the tool is loved by pretty much everyone who uses it, and with Percona at its side, we all know that the tool is going to deliver awesomeness by day and by night.

What is Percona XtraBackup?

As already stated above, Percona XtraBackup is one of the primary offerings for MySQL & Percona database administrators developed by Percona. The tool is an open-source backup utility that does not lock our databases during the backup processes it performs. Percona says that their tool can provide automatic verification of backups that have been taken, offer fast dumping and restore times, and above all, it’s supported by their award-winning consulting services helping us make sure that our data and its backups are in safe hands by day and by night.

How To Use Percona XtraBackup?

In order to start exploring the features offered by Percona’s XtraBackup, please install the tool before proceeding any further – the tool can be installed from a repo, from a tarball, from packages, or via source code – the old-fashioned way of installing the tool from a repository is a favourite option for many. To install the tool from a repo, head over to Percona’s documentation, and once you’re done installing, you can backup your data by issuing the command below (the command below will perform a full backup of your database, and then store the backup in a directory named “backups”):
xtrabackup --backup --target-dir=/backups

Bear in mind that in this case, contrary to mysqldump, the “backups” directory won’t consist of only one file – Percona’s XtraBackup would backup the following:

  • The my.cnf file that consists of the most vital information for MySQL to function correctly (XtraBackup will take a backup of the file and name it backup-my.cnf)
  • The file vital for InnoDB to function correctly – ibdata1. Since ibdata1 holds data, indexes, Multiversion Concurrency Control (MVCC) data, and double write & insert buffers that are necessary for InnoDB to work in the way it does, without it InnoDB’s infrastructure would plummet to ashes.
  • All databases inside of your MySQL infrastructure including the test and performance_schema databases.

Incremental backups can also be made in a very similar fashion: first, take a full backup using the command provided above, then issue a very similar statement, just add a --incremental-basedir statement at the end of it like so:
xtrabackup --backup --target-dir=/backups –incremental-basedir=/incbackups

In order to take a compressed backup, add the --compress option, and for partial backups, use one or more of the following options:

  • Use the --databases or --databases-file options to back up a database or a list of databases from a file:
    xtrabackup --databases=”db1 db2 test_db demo_db”
    xtrabackup --databases-file=databases.txt*
    * The file databases.txt would need to contain databases and tables in the format of database.table.
  • Use the --tables or --tables-file options to back up a table or a list of tables in the same fashion you would back up databases.

Woohoo – you’ve now learned how to take backups using Percona XtraBackup! That’s not everything, though – when using Percona XtraBackup you would also find yourself needing to prepare your backups for them to be successfully restored: we will tell you how to do that in the next section.

Preparing XtraBackups

As you could have noticed, Percona XtraBackup doesn’t take backups in a simple backup.sql form you might be used to when using mysqldump or other database backup tools – instead, Percona XtraBackup often takes a backup of the files associated with the database with itself too (we have covered those in the previous chapter.)

That’s why all of the backups taken using Percona XtraBackup need to be prepared for recovery before they can be successfully recovered – here’s how to do that for each flavour of backups:

  • To prepare a full or an encrypted backup to be restored, run the following command:
    xtrabackup --prepare --target-dir=backups/
  • To prepare an incremental backup to be recovered, you would need to ensure that the rollback phase will be skipped by specifying an --apply-log-only option. Percona themselves state that if the rollback phase isn’t prevented, the incremental backups would be worth nothing and you would have to start over, so keep that in mind. To prepare an incremental backup, run the same command as with full backups, just with the --apply-log-only option, nothing complex here:
    xtrabackup –apply-log-only –prepare –target-dir=backups/
  • To prepare a partial backup, specify the --export tag (don’t worry about warnings in this case – they are most likely issued because InnoDB “sees” tables, but their files do not exist in the data directory):
    xtrabackup --prepare --export --target-dir=backups/

And.. you’re done! Well, kind of. Now you also need to restore the backups, right?

Restoring XtraBackups

Now that you’ve made and prepared your backups, there will obviously be a point in time where you would need to restore what you’ve got in store. Again, such a process is a little different for each backup type, but don’t fret – we’re here to help. Here’s how to come around this issue:

  • To restore full, compressed, incremental, or encrypted backups, add a --copy-back option to restore your backup to the data directory:
    xtrabackup --copy-back --target-dir=/backups
  • To restore partial backups, all you have to do is restore all of the tables in the partial backup (copy them back to the server of your choice.)

That’s it – it’s that simple! Of course, we can run into a couple of issues during these steps as well, so it’s always beneficial to keep an eye out for the documentation.

Options Offered by Percona XtraBackup

As with everything command-line related, Percona XtraBackup has a couple of options associated with itself. Some of them are as follows (all of the options can be found over at the Percona’s documentation):

Percona XtraBackup Option

Meaning

--backup

Takes a backup of the database.

--check-privileges

Checks if Percona XtraBackup has all of the required privileges to be operating properly.

--apply-log-only

Prepares to take incremental backups by ignoring all stages except the redo stage.

--copy-back

Restores a backup. This option is meant to be used in conjunction with other options (see examples above.)

--databases=x | --tables=x

“x” specifies a database or a table inside of a database to be backed up. These options are similar to --databases-file or --tables-file options that backs up databases or tables from a file.

--defaults-file

Makes XtraBackup read only the options specified in the file after this parameter (the file will most likely be my.cnf.)

Summary

Percona XtraBackup is the flagship tool in Percona’s backup arsenal – the tool is widely used by junior and senior database administrators alike and as it avoids the locking of databases during its backup procedures, takes backups in a quick and safe fashion, and allows all kinds of backups to be restored quickly, there’s no doubt that Percona XtraBackup will be the option of choice for database engineers for many years to come. We hope that this blog post has provided you with some of the insight into the Percona’s XtraBackup world and that you will refer to the Percona’s manual for more information, and we will see you in the next blog!

The post Backing Up MySQL Part 2: Percona XtraBackup appeared first on Simple Talk.



from Simple Talk https://ift.tt/6bdQxzR
via

Saturday, November 12, 2022

Introducing the MySQL DELETE statement

Preparing your MySQL environment

As with the previous few articles, I used the same database and tables for the examples in this article (the travel database and the manufacturers and airplanes tables). In this case, however, I recommend that you start from scratch and rebuild the database and tables to keep things simple for this article. To set up the database, run the following script:

DROP DATABASE IF EXISTS travel;

CREATE DATABASE travel;

USE travel;

CREATE TABLE manufacturers (
  manufacturer_id INT UNSIGNED NOT NULL AUTO_INCREMENT,
  manufacturer VARCHAR(50) NOT NULL,
  create_date TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
  last_update TIMESTAMP NOT NULL 
    DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
  PRIMARY KEY (manufacturer_id) )
ENGINE=InnoDB AUTO_INCREMENT=1001;

CREATE TABLE airplanes (
  plane_id INT UNSIGNED NOT NULL AUTO_INCREMENT,
  plane VARCHAR(50) NOT NULL,
  manufacturer_id INT UNSIGNED NOT NULL,
  engine_type VARCHAR(50) NOT NULL,
  engine_count TINYINT NOT NULL,
  max_weight MEDIUMINT UNSIGNED NOT NULL,
  wingspan DECIMAL(5,2) NOT NULL,
  plane_length DECIMAL(5,2) NOT NULL,
  parking_area INT GENERATED ALWAYS AS ((wingspan * plane_length)) STORED,
  icao_code CHAR(4) NOT NULL,
  create_date TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
  last_update TIMESTAMP NOT NULL 
    DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
  PRIMARY KEY (plane_id),
  CONSTRAINT fk_manufacturer_id FOREIGN KEY (manufacturer_id) 
    REFERENCES manufacturers (manufacturer_id) ) 
ENGINE=InnoDB AUTO_INCREMENT=101;

Once you’ve created the database, you can add the sample data you’ll need to follow along with the exercises in this article. Start by running the following INSERT statement to add data to the manufacturers table:

INSERT INTO manufacturers (manufacturer)
VALUES ('Bombardier'), ('Beagle Aircraft Limited');

SELECT *
FROM   manufacturers;

The statement adds two rows to the manufacturers table and outputs those rows. The manufacturer_id column in those rows should have been assigned the values 1001 and 1002.

After you confirm the data in the manufacturers table, you should run the following INSERT statement, which populates the airplanes table:

INSERT INTO airplanes 
  (plane, manufacturer_id, engine_type, engine_count, 
    wingspan, plane_length, max_weight, icao_code)
VALUES
  ('Learjet 24',1001,'Jet',2,35.58,43.25,13000,'LJ24'),
  ('Learjet 24A',1001,'Jet',2,35.58,43.25,12499,'LJ24'),
  ('Challenger (BD-100-1A10) 300',1001,'Jet',2,63.83,68.75,38850,'CL30'),
  ('Challenger (BD-100-1A10) 350',1001,'Jet',2,69,68.75,40600,'CL30'),
  ('Challenger (CL-600-1A11) 600',1001,'Jet',2,64.33,68.42,36000,'CL60'),
  ('Challenger (CL-600-2A12) 601',1001,'Jet',2,64.33,68.42,42100,'CL60'),
  ('A.109 Airedale',1002,'piston',1,36.33,26.33,2750,'AIRD'),
  ('A.61 Terrier',1002,'piston',1,36,23.25,2400,'AUS6'),
  ('B.121 Pup',1002,'piston',1,31,23.17,1600,'PUP'),
  ('B.206',1002,'piston',2,55,33.67,7500,'BASS'),
  ('D.4-108',1002,'piston',1,36,23.33,1900,'D4'),
  ('D.5-108 Husky',1002,'piston',1,36,23.17,2400,'D5');

SELECT *
FROM   airplanes;

The values 1001 and 1002 from the manufacturers table provide the foreign key values for the manufacturer_id column in the airplanes table. After you run the second INSERT statement, the SELECT query will let you confirm that 12 rows have been added to the airplanes table. The first row should have been assigned 101 as the plane_id value, and the plane_id values for the other rows should have been incremented accordingly.

The DELETE statement syntax

The basic syntax for the DELETE statement is fairly straightforward and includes many of the same elements you saw in the other DML statements (INSERT and UPDATE article links):

DELETE [IGNORE] FROM table_name
[WHERE where_condition]
[ORDER BY order_list]
[LIMIT row_count]

The syntax shown here does not include all supported statement components, but it provides the basic elements you need to know to get started with the DELETE statement. You can refer to MySQL topic DELETE Statement for the complete syntax. In the meantime, here’s a breakdown of the statement’s clauses, as I’ve shown in the syntax:

  • The DELETE clause, which includes the FROM subclause, is the only mandatory clause in the DELETE statement. The clause identifies that table from which the data will be deleted. You can specify multiple tables, which involves defining join conditions, but my focus in this article is on single-table deletes. The DELETE clause also supports the use of the IGNORE modifier for returning a warning message, rather than an error, if an issue arises.
  • The WHERE clause determines which rows to delete, based on one or more search conditions. The clause works much like the WHERE clause in SELECT and UPDATE statements. Although the WHERE clause is optional, you should be very careful running a DELETE statement that does not include one. Without a WHERE clause, the statement will delete every row in the target table, unless the LIMIT clause is included.
  • The ORDER BY clause specifies the order that rows should be deleted. This clause is used primarily in conjunction with the LIMIT clause to help better direct which rows should be removed. The ORDER BY clause is similar to the one you saw in the SELECT and UPDATE statements. The clause is optional and cannot be used for multi-table deletes.
  • The LIMIT clause limits the number of rows that will be deleted. When used with the ORDER BY clause, the deleted rows will be determined by the sort order specified by that clause. As with the ORDER BY clause, the LIMIT clause is optional and cannot be used for multi-table deletes.

As you work through the examples in this article, you’ll get a better sense of how the various statement elements work together. That said, the clauses are, for the most part, self-explanatory, and for most uses, you should have little trouble figuring out how they work. The larger concern is that you can lose a lot of data if you’re not careful when using this statement, so always exercise caution, and be sure that all data is fully protected. Above all, make sure you’re not working in a production environment when learning how to use the DELETE statement.

Deleting data from a MySQL table

As noted above, the DELETE clause is the only mandatory clause in a DELETE statement. If you run a DELETE statement with only this clause, it will remove all of the data from the target table—a consideration that should not be taken lightly. If you determine that this is what you want to do, you need only specify the DELETE and FROM keywords, followed by the table name, as in the following example:

DELETE FROM airplanes;

The statement will delete all data from the airplanes table, unless safe mode is enabled. Safe mode is typically enabled by default on a MySQL instance to help limit the possibility of updating or deleting all data in a table.

Much the same as described for the UPDATE statement, if safe mode is enabled on your MySQL instance, you’ll receive the following error message when you try to run the above statement:

Error Code: 1175. You are using safe update mode and you tried to update a table without a WHERE that uses a KEY column. To disable safe mode, toggle the option in Preferences -> SQL Editor and reconnect.

When safe mode is enabled, you cannot run a DELETE statement without a WHERE clause that does not specify a key column in the WHERE clause criteria, unless you include a LIMIT clause. This helps to ensure that you don’t inadvertently delete a table full of data. You can disable safe mode (as the error message suggests) by setting the server properties or by temporarily disabling safe mode when you run the DELETE statement. The temporary approach is usually the safest.

To temporarily disable safe mode, use a SET statement to change the SQL_SAFE_UPDATES system variable to 0 prior to running your DELETE statement and then set the variable to 1 after running the statement, as shown in the following example:

SET SQL_SAFE_UPDATES = 0;

DELETE FROM airplanes;

SET SQL_SAFE_UPDATES = 1;

The SET statements toggle the system variable off and then on during the current session. When taking this approach, be aware that if your DELETE statement generates an error, statement execution will stop and the second SET statement will not execute, so make sure you run this statement to reset the SQL_SAFE_UPDATES variable to 1. Also note that the SET statement supports the optional GLOBAL modifier, which defines a variable at the global scope. However, I recommend that you do not use this option when disabling safe deletes. It is less risky to disable safe mode at the session level to avoid any inadvertent data modifications. Use GLOBAL only if it’s essential in your situation.

Once safe mode is disabled, you should be able to run your DELETE statement without generating an error. After you do, you can confirm your changes with a simple SELECT statement that retrieves all data from the airplanes table. The statement should return no rows.

Adding a WHERE clause to your DELETE statement

In most cases, you’ll want to include a WHERE clause in your DELETE statements so you can target which rows in a table should be deleted (as opposed to deleting all rows). The WHERE clause defines one or more search conditions that specify exactly what data to delete. To see how this works, you should first add the data back to the airplanes table (assuming you’re trying out these examples):

INSERT INTO airplanes 
  (plane, manufacturer_id, engine_type, engine_count, 
    wingspan, plane_length, max_weight, icao_code)
VALUES
  ('Learjet 24',1001,'Jet',2,35.58,43.25,13000,'LJ24'),
  ('Learjet 24A',1001,'Jet',2,35.58,43.25,12499,'LJ24'),
  ('Challenger (BD-100-1A10) 300'
               ,1001,'Jet',2,63.83,68.75,38850,'CL30'),
  ('Challenger (BD-100-1A10) 350'
                ,1001,'Jet',2,69,68.75,40600,'CL30'),
  ('Challenger (CL-600-1A11) 600'
                ,1001,'Jet',2,64.33,68.42,36000,'CL60'),
  ('Challenger (CL-600-2A12) 601'
               ,1001,'Jet',2,64.33,68.42,42100,'CL60'),
  ('A.109 Airedale',1002,'piston',1,36.33,26.33,2750,'AIRD'),
  ('A.61 Terrier',1002,'piston',1,36,23.25,2400,'AUS6'),
  ('B.121 Pup',1002,'piston',1,31,23.17,1600,'PUP'),
  ('B.206',1002,'piston',2,55,33.67,7500,'BASS'),
  ('D.4-108',1002,'piston',1,36,23.33,1900,'D4'),
  ('D.5-108 Husky',1002,'piston',1,36,23.17,2400,'D5');

SELECT *
FROM airplanes;

Confirm that the data has been re-added to the airplanes table by looking at the output of the SELECT statement. The table should now be populated with the same 12 rows, with one notable difference. The first plane_id value is now 113 rather than 101 because MySQL tracks the last auto-incremented value that was assigned to a row, even if that row has been deleted.

After you’ve inserted the data into the airplanes table, you can run the following DELETE statement, which includes a basic WHERE clause:

DELETE FROM airplanes
WHERE icao_code = 'pup';

The search condition specifies that the icao_code value must equal pup for a row to be deleted. However, if you try to run this statement and safe mode is enabled, MySQL will again return error 1175:

Error Code: 1175. You are using safe update mode and you tried to update a table without a WHERE that uses a KEY column. To disable safe mode, toggle the option in Preferences -> SQL Editor and reconnect.

MySQL returns this error because the WHERE clause does not include a key column in its search condition. To get around this, you can again temporarily disable safe mode at the session level:

SET SQL_SAFE_UPDATES = 0;

DELETE FROM airplanes
WHERE icao_code = 'pup';

SET SQL_SAFE_UPDATES = 1;

If you query the airplanes table after running these statements, the table should now include only 11 rows rather than 12. Only one row satisfied the search condition, so only that row was deleted.

In cases where you specifically know what rows to delete, you should try to use a key column in your search condition to avoid having to disable safe mode. For example, the row deleted in the previous example had a plane_id value of 121. As a result, you could have recast the DELETE statement as follows:

DELETE FROM airplanes
WHERE plane_id = 121;

Of course, it’s not always practical to use a key column, in which case, you should define the WHERE clause in a way that best suits your situation, even if it means specifying multiple search conditions, as in the following example:

SET SQL_SAFE_UPDATES = 0;

DELETE FROM airplanes
WHERE engine_type = 'piston' AND max_weight < 2500 ;

SET SQL_SAFE_UPDATES = 1;

Because the WHERE clause contains no key column, safe mode must again be disabled during the session.

The WHERE clause itself includes two search conditions. The first one specifies that the engine_type value must be piston, and the second one specifies that the max_weight value must be less that 2500. The search conditions are connected by the AND logical operator, which means that both conditions must evaluate to true for a row to be deleted.

In this case, several rows matched both search conditions, so they were all removed from the airplanes table. If you query the table, you should find that it now includes only eight rows.

Adding ORDER BY and LIMIT clauses to your DELETE statement

Together, the ORDER BY and LIMIT clauses help you better control how rows are deleted from a table. To see how this works, start be adding a row back into the airplanes table:

INSERT INTO airplanes 
  (plane, manufacturer_id, engine_type, engine_count, 
    wingspan, plane_length, max_weight, icao_code)
VALUES
  ('D.4-108',1002,'piston',1,36,23.33,1900,'D4');

Because this row is being added separately from when the previous rows were added, it will have a different create_date value from the other rows. (The value is a timestamp.) You can use this value to single out the row when building your DELETE statement:

DELETE FROM airplanes
ORDER BY create_date DESC
LIMIT 1;

Notice that the statement includes no WHERE clause with a key column and that there are no SET statements. You can get away with this here because the DELETE statement includes a LIMIT clause. If necessary, you can include a WHERE clause with the ORDER BY and LIMIT clauses, but it’s not needed in this case.

The ORDER BY clause specifies that the rows should be deleted based on the create_date values, sorted in descending order. This ensures that the last row inserted is the first row deleted, assuming it has a unique create_date value. The LIMIT clause then specifies that only one row should be deleted. This will be the first row as it is determined by the ORDER BY clause. You might also consider this approach when deleting archived data, except that you would likely specify an amount other than 1 in the LIMIT clause, such as 100, 1000, or another value.

It’s hard to say how often you’ll use the ORDER BY and LIMIT clauses in your queries. But it’s good to know that you have this option if you want to apply similar logic when deleting data from your database tables.

You can also use these clauses independently of each other. For example, you might use the LIMIT clause when you need to delete a large number of rows from a table and you’re concerned about the impact on database performance. Instead of deleting all the rows at once, you can delete them in batches based on the number of rows specified in the LIMIT clause. Then you can simply rerun the DELETE statement until all the target rows have been removed.

Using the IGNORE modifier in your DELETE statement

As noted earlier, the DELETE statement supports the use of the optional IGNORE modifier, which you also saw in the INSERT and UPDATE statements. When IGNORE is used, your DELETE statement will return a warning rather than an error if an issue arises. In addition, MySQL will continue with the statement execution. If you don’t use IGNORE, MySQL will return an error and stop all statement execution, including any statements in the batch that follow the DELETE statement.

To see how this works, start by running the following INSERT statement, which adds several rows to the manufacturers table:

INSERT INTO manufacturers (manufacturer)
VALUES ('Airbus'), ('Beechcraft'), ('Cessna'), ('Piper');

SELECT *
FROM   manufacturers;

From the output of this batch, you should find that the Airbus row has a manufacturer_id value of 1003. You will use this value as the foreign key value when you add a row to the airplanes table for an Airbus plane. To add the row, run the following INSERT statement:

INSERT INTO airplanes 
  (plane, manufacturer_id, engine_type, engine_count, 
    wingspan, plane_length, max_weight, icao_code)
VALUES
('A220-100',1003,'Jet',2,115.08,114.75,134000,'BCS1');

Suppose you now want to delete all the rows you recently added to the manufacturers table. You might try to run the following DELETE statement, using the manufacturer_id values for those rows:

DELETE FROM manufacturers
WHERE manufacturer_id IN (1003, 1004, 1005, 1006);

When you try to run this statement, MySQL stops statement execution and returns the following error, which indicates you have a foreign key violation:

Error Code: 1451. Cannot delete or update a parent row: a foreign key constraint fails (`travel`.`airplanes`, CONSTRAINT `fk_manufacturer_id` FOREIGN KEY (`manufacturer_id`) REFERENCES `manufacturers` (`manufacturer_id`))

MySQL returned this error because you tried to delete a row that was being referenced by the airplanes table. As a result, the entire statement failed and no rows were deleted. However, you can ensure that the DELETE statement continues to execute even if one of the deletions fails by including the IGNORE modifier:

DELETE IGNORE FROM manufacturers
WHERE manufacturer_id IN (1003, 1004, 1005, 1006);

Now the statement returns the following message, which shows the number of rows that have been affected and provides a warning:

3 row(s) affected, 1 warning(s): 1451 Cannot delete or update a parent row: a foreign key constraint fails (`travel`.`airplanes`, CONSTRAINT `fk_manufacturer_id` FOREIGN KEY (`manufacturer_id`) REFERENCES `manufacturers` (`manufacturer_id`))

From this message, you can see that three rows have been deleted and that there has been a foreign key violation. If you query the manufacturers table, you’ll find that it now contains only three rows, including the one for Airbus. To remove all the Airbus data, you must first delete any referencing records from the airplanes table:

DELETE FROM airplanes
WHERE manufacturer_id = 1003;

You should then be able to run the following DELETE statement to remove the Airbus record from the manufacturers table, which will leave you with only two rows in that table:

DELETE  FROM manufacturers
WHERE manufacturer_id = 1003;

The IGNORE modifier can be useful when you need to schedule a deletion and want the statement execution to continue even if some rows cannot be deleted. This could be especially useful when deleting large sets of data. After the bulk of records have been deleted, you can go back and address any warnings. Be aware, however, that the IGNORE modifier works for only certain types of errors.

Working with the MySQL DELETE statement

The DELETE statement, along with the SELECT, INSERT, and UPDATE statements, represent four of the most important statements you’ll use when working with MySQL data. However, they’re not the only DML statements. MySQL also supports DML statements such as CALL, LOAD DATA, REPLACE, and TABLE.

But the four we’ve covered so far in this series are a great place to start for manipulating data, with the DELETE statement helping to complete that picture. The statement makes it possible to easily remove data that is incorrect or outdated. In fact, the statement can be almost too easy to use, and you must be careful not to inadvertently delete the wrong data. That said, the DELETE statement is extremely useful, and you should be sure that you fully understand how to use it, along with the SELECT, INSERT, and UPDATE statements.

 

The post Introducing the MySQL DELETE statement appeared first on Simple Talk.



from Simple Talk https://ift.tt/8lqiUrj
via

Friday, November 11, 2022

Optimizing MySQL: The Basics of Query Optimization

MySQL is a very interesting beast to deal with – both junior developers and expert database administrators working with the RDBMS know that the world of MySQL offers a breadth of opportunities not found anywhere else: people can choose a storage engine to use from a list of powerful storage engines, the RDBMS can be heavily optimized to serve very specific needs, MySQL has multiple specific types of tools and operations in place that can be used to optimize query performance, etc.

One of the most frequent causes of headaches for database administrators for ages are queries and their optimization for speed. As old as this question is, it’s still very hot. Have a skim through database-related questions posted on StackOverflow within the present week – do you see any patterns? “Slow query…”, “Performance Improvement…“, “Indexing strategy for..”, “Preventing toxic selects”, “Database not using index.” All of those questions mean one thing – issues concerning database performance are still very important, and people want to know how best to work with their queries to achieve the best mix of performance, availability, security, and capacity.

In the upcoming series, we’re focusing on the optimization of MySQL – we’re going to walk you through storage engine and query optimization, we’re going to tell you a couple of secrets situated around MySQL optimization settings, how best to measure performance, amongst other things. We’re starting small – in this blog, we’re walking you through the basics of the basics and answering the age-old question – why are my queries slow?!

Why Your Queries Are Slow

MySQL offers a couple of types of queries we can choose from:

  • INSERT queries that insert data into our database;
  • SELECT queries that read through data;
  • UPDATE queries that update our data;
  • DELETE queries that delete data within our database;
  • ALTER queries that change (alter, hence the name) data within our database.

The main reasons those queries are slow are as follows:

  • INSERT queries are slow most likely because there is a lot of data to be inserted into the database or (and) because the database isn’t optimized properly.
  • SELECT queries are slow mainly because they read through a lot of unnecessary data. The less data there is to scan through, the faster our SELECT queries will become. That’s it – there’s no rocket science involved.
  • UPDATE queries are usually slow because we either have an index on the column that is being updated or we are executing a huge update query without familiarizing ourselves with the internals of storage engines – InnoDB usually locks the table when UPDATEs are being performed and unlocks it after the fact.
  • DELETE queries might be slow because of an index on the column that we’re deleting the data from or because we’re deleting massive amounts of data without knowing the internals of storage engines as well. However, unlike UPDATE queries, DELETE queries have a couple of tricks up their sleeve – they can be massively sped up if we use the TRUNCATE query to remove all of the rows at once, then reimport only the necessary data.
  • ALTER queries are usually slow because of improperly optimized operations that run internally:
    • ALTER queries first make a copy of the table.
    • Once the copy of the table is made, the data existing in the original table is copied into it.
    • All of the necessary changes are made.
    • The original table is swapped with the copy.

All of those operations consume time – time that must be accounted for.

Some of such operations can be optimized for performance by properly adjusting the buffer pool used by the InnoDB storage engine, others can be optimized by adding or removing the presence of indexes and partitions, and all of them can be optimized by understanding the internals of queries available within MySQL. For now, though, understand one thing – queries are processes comprised of tasks, and to optimize the performance of those processes, we must optimize the tasks within those processes.

We hope that you’ve enjoyed reading this article and that it helped you grasp the basics of query optimization in the database management system of MySQL – stay tuned to learn how to optimize tasks within processes, read our blog to learn about database management systems in more detail, and until next time.

The post Optimizing MySQL: The Basics of Query Optimization appeared first on Simple Talk.



from Simple Talk https://ift.tt/wniH16e
via

Monday, November 7, 2022

The Basics of Deleting Data from a SQL Server Table

Entry in Greg Larsen's series on Learning T-SQL. The series so far:

  1. The basic T-SQL Select Statement
  2. Ordering a result set in SQL Server
  3. The ins and outs of joining tables in SQL Server
  4. Summarizing data using GROUP BY and HAVING
  5. The Basics of Inserting Data into a SQL Server Table
  6. The Basics of Updating Data in a SQL Server Table

Over time data in SQL Server tables needs to be modified. There are two major different aspects of modifying data: updating and deleting. In my last article “Updating SQL Server Data” I discussed using the UPDATE statement to change data in existing rows of a SQL Server table. In this article I will be demonstrating how to use the DELETE statement to remove rows from a SQL Server Table.

Syntax of the Basic DELETE statement

Deleting data seems like a simple concept so you would think the syntax for a DELETE statement would be super simple. In some respects that is true, but there are many ways and aspects of how the DELETE statement can be used. In this article I will be discussing only the basic DELETE statement.

The syntax for basic DELETE statement can be found in Figure 1:

DELETE

[ TOP (<expression>) [ PERCENT ] ]

[ FROM ] <object>

[ WHERE <search_condition>]

Figure 1: Basic syntax of the DELETE statement

Where:

  • expression – Identifies the number or percentage of rows to be delete. When only the TOP keyword is identified the expression identifies the number of rows to be deleted. It the keyword PERCENT is also included then the expression identifies the percentage of rows to be delete.
  • object – Identifies the table or view from which rows will be deleted.
  • search_condition – Identifies the criteria for which a row must meet in order to be deleted. The search_condition is optional. When it is excluded from a DELETE statement, then all the rows in the object will be deleted.

For the complete syntax of the DELETE statement refer to the Microsoft Documentation which can be found here.

To better understand the syntax of the simple DELETE statement and how to use it, several examples are provided below. But before these examples can be run a couple sample tables need to be created.

Sample Data

Listing 1 contains a script to create two sample tables in the tempdb database.

USE tempdb;
GO

CREATE TABLE dbo.LoginTracking (
ID INT IDENTITY(1,1),
LoginName varchar(100),
LoginDataTime datetime,
LogoffDateTime datetime);
GO

INSERT INTO dbo.LoginTracking VALUES
('Scott','2022-07-11 08:41:31','2022-07-11 11:45:50'),
('Sally','2022-07-11 08:55:27','2022-07-11 11:59:59'),
('Dick','2022-07-11 09:05:17','2022-07-11 16:15:37'),
('Dick','2022-07-12 08:05:11','2022-07-12 15:50:31'),
('Scott','2022-07-12 08:12:27','2022-07-12 16:11:22'),
('Sally','2022-07-12 09:20:06','2022-07-12 16:45:11'),
('Dick','2022-07-13 08:10:13','2022-07-13 15:59:45'),
('Scott','2022-07-13 08:12:37','2022-07-13 16:21:38'),
('Sally','2022-07-13 09:07:05','2022-07-13 16:55:17');
GO

CREATE TABLE dbo.LoginsToRemove(
LoginName varchar(100));
GO

INSERT INTO dbo.LoginsToRemove VALUES ('Sally');
GO

Listing 1: Script to create sample tables.

The first table created in Listing 1 is the dbo.LoginTracking table. This table is the table from which rows will be deleted. The second table created is dbo.LoginsToRemoved. This table is a staging table that contains a single column named LoginName. This table will be used to show how to delete rows using rows in another table.

If you would like to follow along and run the code in this article you can use Listing 1 to create the two sample tables in the tempdb database on your instance of SQL Server.

Deleting a Single Row

To delete a single row, the DELETE command needs to identify the specific row that needs to be deleted. This is done by including a WHERE constraint. The WHERE constraint needs to uniquely identify the specific row to delete. The code in Listing 2 will delete the LoginTracking record that have a LoginName “Scott” and an ID value of 1.

USE tempDB;
GO

DELETE FROM dbo.LoginTracking
WHERE LoginName = 'Scott' and ID = 1;
GO

Listing 2: Deleting a single row

In the LoginTracking table there are multiple rows that have a LoginName of “Scott”. Therefore the ID column value of “1” was also included in the search_condition to uniquely identify which single row was to be deleted.

Only the ID column alone could have been used to identify the single record to be deleted. However, in many cases having extra filter conditions can be useful. For example, if the row where ID = 1 has a different LoginName, it would not be deleted.

Note: When you are expecting a certain number of rows to be deleted, it is a good idea to check the value in @@ROWCOUNT to make sure.

Deleting Multiple Rows

A single DELETE statement can also be used to delete multiple rows. To delete multiple rows the WHERE clause needs to identify each row in the table or view that needs to be deleted. The code in Listing 3 identifies two LoginTracking rows to delete by identifying the ID column values for the rows to be deleted.

USE tempDB;
GO

DELETE FROM dbo.LoginTracking
WHERE ID = 2 or ID=3;
GO

Listing 3: Deleting multiple rows with a single DELETE statement

By specify a WHERE clause, that identifies multiple rows the code in Listing 2 will delete multiple rows in the LoginTracking table when this code is executed.

Using the TOP Clause to Delete Rows of Data

When the TOP clause is included with a DELETE statemen, it identifies the number of random rows that will be deleted from the table or view being referenced. There are two different formats for using the TOP clause. (1) identifies the number of rows to be delete.

The code in Listing 4 shows how to use the TOP clause to delete one random row from the LoginTracking table.

USE tempdb;
GO

DELETE TOP(1) FROM dbo.LoginTracking; 
GO

Listing 4: Deleting one row using the TOP clause

The reason the top clause deletes random row is because SQL Server does not guarantee the order rows will be returned, without an ORDER clause.

If you need to delete rows in a specific order, it is best to use a subquery that uses TOP and an order by, as shown in Listing 5.

USE tempdb;
GO

DELETE TOP (2) FROM dbo.LoginTracking
WHERE ID IN 
   (SELECT TOP(2) ID FROM dbo.LoginTracking ORDER BY ID DESC);
GO

Listing 5: Deleting the last 2 rows based in ID

Executing the code in Listing 5 deletes the last two rows in the LoginTracking table, based on the descending order of the ID column. The code accomplished this by using a subquery in the WHERE constraint. The subquery uses the TOP clause to identify the two ID values that will be deleted based on descending sort order. This list of ID values is then used as the WHERE filter condition to identify the two rows to be delete.

Note: in this case, the TOP (2) in the DELETE is technically redundant. But if the ID column didn’t contain unique values, and the TOP clause would need to be included to make sure only two rows would have been deleted,.

The TOP clause in the prior two examples identified a specific number of rows to be deleted. The TOP clause also supports identifying a percentage of rows to delete. Before showing an example of deleting a percentage of rows using the TOP clause let’s first review the details of the rows still in the LoginTracking table, by running the code in the Listing 6.

USE tempdb;
GO

SELECT * FROM dbo.LoginTracking;

Listing 6: Displaying the rows currently in the dbo.LoginTracking table

When the code in Listing 6 is run the results in Report 1 are displayed.

Text Description automatically generated

Report 1: Current rows in the LoginTracking table.

Currently there are 5 rows in the LoginTracking table.

To delete a percentage of the rows in a table the TOP clause is use along with the PERCENT keyword. The expression provided provide with the TOP clause identifies the percentage of rows to delete. The code in Listing 7 specifies 41 percent of the rows in the LoginTracking table should be deleted.

USE tempdb;
GO
DELETE TOP(41) PERCENT FROM dbo.LoginTracking;
GO
SELECT * FROM dbo.LoginTracking;
GO

Listing 7: Deleting 41 percent of the rows using the TOP clause.

When Listing 4 is executed the output in Report 2 is displayed.

Graphical user interface, text Description automatically generated

Report 2: Number left in LoginTracking after the code in Listing 7 was run

By reviewing Report 2 and comparing it with Report 1 results, you can see 3 rows were deleted, which means 60 percent of the rows were deleted. Why 60 percent? The reason 60 percent of the rows were deleted is because SQL Server needs to delete a percentage of rows that equates to a whole number. Since 41 percent of 5 rows would have been 2.05 of the rows in the LoginTracking tracking table, SQL server rounded the row count up to 3. 3 equates to 60 percent of the rows when it performed the DELETE statement.

Using a Table to Identify the Row to Delete

There are times when you might need to identify rows to delete based on a table. To show how to delete rows based on rows in a table, the LoginsToRemove table was created in Listing 1. The LoginsToRemove table could be considered a staging table, which identifies the logins that need to be deleted. The code in Listing 8 uses this table to delete rows from the LoginTracking table.

USE tempdb;
GO

DELETE FROM dbo.LoginTracking
FROM dbo.LoginTracking JOIN dbo.LoginsToRemove
ON LoginTracking.LoginName = LoginsToRemove.LoginName
GO

Listing 8: Using a table to identify rows to be deleted from a table

The code in Listing 8 joined the LoginsToRemove table with the LoginTracking table on the LoginName column from each table. This join operation only finds those rows in the LoginTracking table which have a matching LoginName in the LoginsToRemove table. In this example all the records that had “Sally” as a LoginName got deleted from the LoginTracking table.

Deleting all Rows from a Table

The DELETE statement can be used to remove all the rows in a table. To accomplish this a DELETE Statement without a WHERE constraint can be executed, as in Listing 9.

USE tempdb;
GO

DELETE FROM dbo.LoginTracking;
GO

Listing 9: Deleting all the rows in a table.

Even though the DELETE statement can delete all the rows in a table, as done in Listing 9, this is not the most efficient way to delete all the rows in a table. The DELETE statement performs a row-by-row operation to delete all the rows. Every time a row is deleted information needs to be written to the transaction log file, based on the database recovery model option. If there are a lot of rows in a table, it could take a long time and use a lot of resources to delete all the rows using a DELETE operation.

An alternative method to remove all rows in a table

A more efficient, though more limited method to delete all rows is to use the TRUNCATE TABLE statement as shown in Listing 10.

USE tempDB;
GO

TRUNCATE TABLE dbo.LoginTracking;
GO

Listing 10: Deleting all the rows in a table using a TRUNCATE TABLE statement

When a TRUNCATE TABLE statement is used, less information is logged to the transaction log file, as well as has some other significate differences. One of those differences is when a DELETE statement is used, if all the rows are deleted from a physical DATA page the page is left empty in the database, which wastes valuable disk space. Additionally, a TRUNCATE TABLE option requires less locks to remove all the rows.

Another important difference is how these two different delete methods affect the identity column’s seed value. When a TRUNCATE TABLE statement is executed the seed value for the identity column is set back to the seed value of the table. Whereas the DELETE statement retains the last identity column value inserted. This means when a new row is inserted, after all the rows were deleted, using a DELETE statement, that the next identify value will not start at the seed value for the table. Instead, the new row’s identity value will be the determined based off the last identity value inserted, and the increment value for the identity column. These differences need to be kept in mind when deleting rows using the DELETE statement during testing cycles.

There are two major limitations to consider. First, the table being truncated cannot be referenced in a FOREIGN KEY constraint. The second is security. To be able to execute a DELETE statement that references a table, you need DELETE permissions. To execute TRUNCATE TABLE you need ALTER TABLE rights, which gives the user the ability to change the table’s structure. For more details on TRUNCATE TABLE, check the Microsoft documentation here.

Deleting data using a view

Deleting from a view is the same as deleting from a table, with one caveat. The delete can only affect one table. This essentially means that the FROM clause of the view can only reference one table to be the target of a DELETE statement. You could have conditions (like subqueries) that reference other objects, but no joins. For example, in listing 11, consider the two view objects.

CREATE VIEW dbo.v_LoginTracking
AS
SELECT ID,
       LoginName,
       LoginDataTime,
       LogoffDateTime
FROM   dbo.LoginTracking;
GO

CREATE VIEW dbo.v_LoginTracking2
AS
SELECT LoginTracking.ID,
       LoginTracking.LoginName,
       LoginTracking.LoginDataTime,
       LoginTracking.LogoffDateTime
FROM dbo.LoginTracking JOIN dbo.LoginsToRemove
ON LoginTracking.LoginName = LoginsToRemove.LoginName
GO

Listing 11: View objects to demonstrate how deleting from a view works

Executing a DELETE statement in the first view: dbo.v_LoginTracking, will work. But attempt to delete rows using the second view: dbo.v_LoginTracking, and you will get the following error.

View or function 'dbo.v_LoginTracking2' is not updatable because the modification affects multiple base tables.

Deleting Rows from a SQL Server table

The DELETE statement is used to delete rows in a SQL Server table or view. By using the DELETE statement, you can delete one or more rows from an object. If all the rows in a table need to be deleted, a DELETE statement is less efficient then using the TRUNCATE TABLE statement, but has fewer limitations and doesn’t reset the identity value information.

Knowing how to use the basic DELETE statement, and when not to use it is important concept for every TSQL programmer to understand.

 

The post The Basics of Deleting Data from a SQL Server Table appeared first on Simple Talk.



from Simple Talk https://ift.tt/sq5zk1R
via

Saturday, November 5, 2022

How to Use Any SQL Database in GO with GORM

Object Relational Mapping is a database abstraction technique that aids developers in manipulating and interacting with SQL databases using the data types provided in the programming language.

ORMs (Object Relational Mappers) are tools (libraries, modules, packages) that provide the functionality for interacting with SQL databases. Popular ORMs are the Prisma ORM for JavaScript, TypeScript, Hibernate for Java, and SQLAlchemy for Python.

Aside from the benefit of interacting with SQL databases in a more natural manner than writing raw SQL, there are many other benefits of using ORMs. Without worrying too much about compatibility, you can use any databases supported by an ORM with the same code (when using common functionality, not every feature will be available for every database type.)

GORM is the most popular ORM in the Go ecosystem. GORM (Go-ORM) is a developer-friendly, full-featured, Code-first ORM for interacting with SQL databases in Go. GORM provides drivers and functionalities like associations, automigration, SQL building, logging, and hooks for database operations, and support for popular SQL databases including MySQL, SQLite, PostgreSQL, Microsoft SQL server, and many database drivers.

Getting Started with GORM

You’ll need to meet a few requirements to get the most out of this article.

  • You have experience working with Go and have Go installed on your machine.
  • You have experience working with SQL and SQL databases.

In this tutorial, you’ll learn how to use GORM with an SQLite database. The processes are generally the same for all other supported SQL databases.

After creating a Go workspace, Install the GORM package using these commands in your working directory.

go get gorm.io/gorm

You’ll need a database driver to work with GORM. Fortunately, GORM also provides database drivers for the popular SQL databases. Install the database driver for your preferred database using any of these commands.

go get gorm.io/driver/sqlite //SQLite 
go get gorm.io/driver/mysql // MySQL 
go get gorm.io/driver/postgres //PostgreSQL 
go get gorm.io/driver/sqlserver //MSSQL 
go get gorm.io/driver/clickhouse //click house

GORM also provides functionalities for using custom database drivers. This is beyond the scope of this article, but you can read more here.

After installing your database driver, you can import the driver for use.

import ( 
        "gorm.io/gorm" 
        _ "github.com/mattn/go-sqlite3" 
      )

As shown above, you’ll have to import the database driver for side effects. (This allows interactions with the database over GORMs interface to do things like query and modify data.)

Connecting to Databases using GORM

After installing and importing the GORM package and your preferred database driver, you can proceed to connect to your database.

Use the Open method of the gorm module to connect to a database. The Open method takes in the connection and configuration methods as input and returns the database connection instance.

db, err = gorm.Open(sqlite.Open("Blogs.db"), &gorm.Config{})
 
if err != nil { 
     panic("failed to connect database") 
}

The previous code connects to an in-memory SQLite database in the code example above. For SQLite, the Open method creates a database in your working directory if the database doesn’t exist. The process is similar if you connect to a database with a connection string. In the following code, I am connecting to a MySQL database using a connection string.

Db, err =        gorm.Open(mysql.Open(“root:admin@tcp(127.0.0.1:3306)/yourdatabase?charset=utf8mb4&parseTime=True&loc=Local”), &gorm.Config{}) 
if err != nil { 
   panic(“failed to connect database”) 
}

Connecting to a database is similar if you use a custom database driver. You must connect to the database based on the specifications that were included in the driver, and pass the connection details to the Open method.

Db, err := gorm.Open(“sqlite3”, “./app.db”)

In this case, you’re using the custom SQLite driver imported above. On a successful database connection, you’ll be able to interact with the database with Go data types in the same way.

Mapping Go Types With GORM

GORM is a code-first ORM. This means you won’t have to define your schema in SQL. For GORM, you use Go structs to describe the schema and column types with GORM tags.

Type Human struct { 
   Age int 
   Name string 
}

On migration, the Human struct will be translated to the database schema. The table’s name in the database will be Human and the Age and Name fields are the column names.

GORM provides tags for adding SQL constraints like primary key, foreign keys, index, not null and other database constraints. Here’s how you can add a GORM tag to a struct.

type Human struct { 
    Age int `gorm:"primaryKey"` 
    Name string `gorm:"size:255;not null;unique"` 
}

The Age field has a primary key constraint, and the Name field has size, not null and unique constraints.

For easy schema modelling, GORM provides the GORM Model. The GORM model is a struct that contains popularly used fields like ID as the primary key and the created, updated, and deleted time as columns. You can use the Model struct by passing the model to your struct.

type Human struct { 
   gorm.Model 
   Age int `gorm:"primaryKey"` 
   Name string `gorm:"size:255;not null;unique"` 
}

The Human struct above evaluates to the HumanPlus struct below in the database on migration since the Human struct inherits the gorm.Model struct.

type HumanPlus struct { 
   Age int `gorm:"primaryKey"` 
   Name string `gorm:"size:255;not null;unique"` 
   ID uint `gorm:"primaryKey"` 
   CreatedAt time.Time 
   UpdatedAt time.Time 
   DeletedAt gorm.DeletedAt `gorm:"index"` 
}

You can embed your custom structs in other structs using the embedded tag.

type Human struct { 
   Age int `gorm:"primaryKey"` 
   Name string `gorm:"size:255;not null;unique"` 
} 
type Beings struct { 
   human Human `gorm:"embedded"` 
   gorm.Model 
}

The Beings struct eventually evaluates to the Evaluate struct below on migration.

type Evaluate struct { 
   Age int `gorm:"primaryKey"` 
   Name string `gorm:"size:255;not null;unique"` 
   ID uint `gorm:"primaryKey"` 
   CreatedAt time.Time 
   UpdatedAt time.Time 
   DeletedAt gorm.DeletedAt `gorm:"index"` 
}

GORM supports most features you’ll want in an ORM, including functionality to set field permissions for more data control. Here’s the complete reference from the GORM documentation.

type User struct { 
   Name string `gorm:"<-:create"` // allow read and create 
   Name string `gorm:"<-:update"` // allow read and update 
   Name string `gorm:"<-"` // allow read and write (create and update) 
   Name string `gorm:"<-:false"` // allow read, disable write permission 
   Name string `gorm:"->"` // readonly (disable write permission 
                           //unless it       configured) 
   Name string `gorm:"->;<-:create"` // allow read and create 
   Name string `gorm:"->:false;<-:create"` // create only (disabled read 
                                                          //from db) 
   Name string `gorm:"-"` // ignore this field when writing and 
                          //reading with struct 
   Name string `gorm:"-:all"` // ignore this field when writing, 
                              //reading and migrating with struct 
   Name string `gorm:"-:migration"` // ignore this field when migrate 
                                    //with struct 
}

Setting up Automigrations With GORM

Automigrations are one of the significant benefits of using ORMs, making it easy to add data entries. You can migrate instances of the struct model using the AutoMigrate method of the database instance.

err = db.AutoMigrate(&Human{}) 
if err != nil { 
    panic("migration failure") 
}

By referencing the struct here, you’ve set the Human struct for auto migration. The AutoMigrate method returns an error that you can handle if any.

Conventionally, you’ll use the function that returns your database instance to set up auto migrations and return the database instance.

func database(DNS string) *gorm.DB { 
   var db *gorm.DB 
   db, err = gorm.Open(mysql.Open(DNS), &gorm.Config{}) 
   if err != nil { 
      panic("failed to connect database") 
   } 
   err = db.AutoMigrate(&Human{}) 
   if err != nil { 
      return 
   } 
      return db 
   }

Inserting Into a Database With GORM

The Create method accepts a reference to the struct initialization and inserts a new row to the database.

person := Human{ 
   Model: gorm.Model{}, 
   Age: 18, 
   Name: "James Dough", 
} 
db.Create(&person)

Depending on your operation; you can use many methods and functions on the Create method.

Using the RowsAffected method on the Create method returns the number of Rows affected.

rows := db.Create(&person).RowsAffected 
log.Println(rows)

There are many other methods you can use with the Create method. In figure1 you can see a few of them.

Graphical user interface, text Description automatically generated with medium confidence

Figure 1 – Additional methods you can use with the Create method.

Reading From a Database With GORM

Reading from a database with GORM is easy, and there are many methods you can use based on the operation.

The read methods decode the query result into an instantiated struct.

type person := new(Human)

The Take , First , and Last methods return a single row. The Take method returns a random row that meets the criteria; the First method returns the first entry that satisfies the query ordered primary key, and the Last method returns the last entry that satisfies the query ordered primary key.

db.Take(&person, "John") 
db.First(&person, "Jane") 
db.Last(&person, "Dough")

Using the Find method, you can also query for all the rows that meet the criteria.

db.Find(&person, "James")

You can use the Where method with any of the Find , First , Take and Last methods for complex queries.

db.Where("Name = ? AND Age >= ?", "James", "22").Find(&person)

Updating Database Entries With GORM

Update operations are easy and quite similar to other GORM operations, and all the methods and functions on read operations are also available.

You can update a column using the Update method after referencing the struct table you want to update using the Model method.

var rows = db.Model(&Human{}).Update("Name", "Junior").RowsAffected

After a successful update operation, the rows variable will store the number of rows affected by the operation.

You can also update multiple columns using the Updates method. The Updates method takes in an initialized struct.

person := Human{ 
    Name: "James", 
    Age: 18, 
} 
db.Model(&person).Updates(person)

Like the read operation, you can use the Where method with any update methods for complex operations.

db.Model(&Human{}).Where("Age = ?", 19).Update("Name", "Junior")

Deleting From a Database With GORM

You can use the Delete method to delete rows in a database. The Delete method takes in the instantiated struct type.

db.Delete(&person)

You can also use the Delete method on the Where method for complex operations

db.Where("name = ?", "John").Delete(&person)

If you need to delete specific columns, You can drop a column using the DropColumn method of the Migrator method of the database instance.

err := db.Migrator().DropColumn(&Human{}, "Name") 
if err != nil { 
return 
}

Here you dropped the Name column of the Human table. The DropColumn method returns an error if there’s any.

The GORM SQL Builder

One of the cons of using ORMs, in general, is the abstraction ORMs provide over the database, reducing the power and flexibility of interacting with databases. Most ORMs like GORM provide SQL builders for raw SQL queries and operations.

You can use the Raw method to write raw SQL queries. The methods on the Raw method serve various functionalities. The Scan method returns the result of the query into the instantiated struct.

type Result struct { 
   ID int 
   Name string 
   Age int 
}
 
var output Result 
db.Raw("SELECT id, Name, Age FROM Human WHERE Name = ?", "James").Scan(&output)

In this case, GORM would decode the output of the SQL query into the output variable You can also execute SQL statements using the Exec method of your database instance.

db.Exec("DROP TABLE Human")

Conclusion

GORM provides most of the functionalities you’ll need in an ORM while prioritizing efficiency and security, making the library every Go developer’s choice for interacting with SQL databases.

In this tutorial, you learned about the GORM library to increase productivity while interacting with databases with Go. You can perform many more operations with the GORM library, and this article has given you an overview. You can check out GORM’s documentation to learn more about the library.

 

The post How to Use Any SQL Database in GO with GORM appeared first on Simple Talk.



from Simple Talk https://ift.tt/TliFZgu
via

Wednesday, November 2, 2022

Calling all leaders: People matter

At the turn of the millennium, I was a young adult, just finished education and starting my very first job at a software company. I was excited, enthusiastic, and felt well-prepared for a life of “Office 9 to 5” and was looking forward to earning my keep. But I was naïve when it came to understanding how people work together in an organization. Here was no distinguished professor to teach lessons, no defined curriculum to structure lessons around, and no predictability about what those lessons might even be. There were unwritten rules, underlying currents, politics, and stratagems aplenty. In an office surrounded by thousands of colleagues, I was still on my own.

Twenty or so years later, I am happy to say that I have kept most of my energy and enthusiasm, while also learning a thing or two about how corporate culture works. I have learned many lessons: Communication is important. Context is key. Expertise earns respect. Boldness, curiosity, and experimentation can be admirable yet intimidating. But my biggest lessons have been that everyone works differently and people’s feelings matter. Nothing highlights this better than the contrast between two different companies I went on to work with. Both were small companies with less than 30 employees. Both were family-owned and run. Both were in the same geographic area and both benefited from the same economic advantages.

Company A treated its employees poorly. We were ordered to follow instructions and discouraged from challenging anything. “Write this document” meant the document had to be written, regardless of whether it was necessary or not, whether anyone would read it or not, or whether there was a better way to do something or not. Senior managers were never present to hear from us: they declined meeting requests, gave brief answers to text messages, and often went off on a tangent when asked for clarification about any topic. We were expected to work far beyond the hours we were paid for, and to be at our desk at 08:30 and still be there until 18:30. There were other things: micromanagement was off the scale, all communication had to take place by email to “maintain records” because there was a culture of finger pointing and blame shifting, customer complaints were ignored, timesheets had to be documented to the minute, and getting appreciated for good work was like drawing blood from stone.

I lasted six months before I got out.

Company B was an employee haven. We were treated with respect. Our views were sought, heard, and frequently followed. We were encouraged to have interests outside of our work area of expertise, to explore, and to self-develop. Senior managers were always available to listen to our problems and help us find our own solutions. We had regular team events, Friday evening drinks, and even a small bonus at Christmas. Company meetings always highlighted team achievements and celebrated our successes. Managers were flexible with work timings, and we willingly volunteered to cover for each other should someone need to leave office during work hours.

I stayed two years and would probably never have left had I not moved to another country.

The first company’s leadership made my colleagues and me feel frustrated, mistrustful, and self-serving. The result was that there was never a spirit of team or community.

The second company’s leadership made us feel valued and respected. The result was that we worked as a team and did our best to meet our customers’ expectations, to delight them and grow the company’s business.

Company A has now gone bust. Company B has grown into a multi-subsidiary group with headquarters in a major European capital city and branches in other locations.

It is people who make the organization. Even history has many examples, from dictators to presidents, some of the unlikeliest folk to have become leaders. Yet, they did.

I believe people follow people, not intangible “ideals.”

 

The post Calling all leaders: People matter appeared first on Simple Talk.



from Simple Talk https://ift.tt/g5JwTuX
via

Saturday, October 29, 2022

The PASS Sessions I Am Most Excited For

The PASS Data Community Summit is coming up in less than three weeks, so it is time to start finalizing your schedule. My schedule this year is fixed, as I will be working behind the scenes helping to manage one of the live online tracks. But that doesn’t stop me from checking the sessions I will be making sure I watch after the show is over if I don’t see them live.

Note: The sessions mentioned are based on things that Louis Davidson (me!) is interested in and doesn’t say anything about sessions I am not mentioning. While I now edit the Simple-Talk website, I will continue to be a data architect/programmer at heart. Hence my preferred sessions are going to be heavily T-SQL programming centric when I am choosing for me.

While I am Producing

Just looking at the list of sessions I am pretty sure I will be involved with, there are some sessions that I want to see either way. For example:

Note: you will need to register/login to follow these links. If you haven’t registered, time is running out! Go to PASS Data Community Summit and register.

In-Memory OLTP Design Principles by Tosten Strauss. I have been fascinated by the memory-optimized objects in SQL Server for many years, though I haven’t used the feature even in a demo manner for a while. I have built quite a bit of sample code with it, but always love to hear more.

SQL Server Table Partitioning – DOs and DON’Ts by Margarita Naumova. Partitioning is not something I have ever done in a production setting, and it is really one of the features in SQL Server that I have never even written very much demo code on. As such it is a topic I always wanted to know more about for that day when I need to implement it (or tech edit a blog about it!).

SSIS Custom Pipeline Component: A Step-by-Step Guide by Arne Bartels. I have spent a lot of time over the past 15 years building SSIS Packages, and I will probably continue to build them to more data around in my personal projects as well.

While I may not professionally build SSIS packages now (and maybe not ever), sometimes it is fun to go to a session and learn all the things you could have done better.


Does this mean I won’t enjoy An Introduction to S3 Data Lake for SQL Server 2022 by Chris Adkin, Getting started with Power BI Deployment Pipelines by Akshata Revankar., Better Data Governance with Purview by Kelly Broekstra, or any of the other sessions I will be in? If you think that, you may have skipped the introduction to this blog. Clearly the answer is “no”. In fact, since I will just be where I am told, I have only barely looked at the schedule after it posted. The sessions may have had a time change since last I looked.

The three highlighted sessions are ones I would have in my list to stream after the conference anyhow!

Saving for Later

I have scanned the schedule a few times and have quite a few more sessions that would be not-miss, front row attenders if I was in Seattle. Of course, the reality is that even if I was there in-person, I still probably would have missed a few. Sometimes it is because there are two sessions that overlap each other (the committee tries to not let that happen for sessions that suit a certain persona (like database programmer), but there are only so many slots to hold sessions. The far more common reason I tend to miss sessions is that I know at least 100-300 people that will be at the Summit… that’s a lot of catching up to do (especially since I haven’t been to a conference in almost three years)!

There are plenty of sessions I will do my best to watch after the week is over. It is aways a bit difficult to find the time to go back and watch every session you want to, but there are a few sessions that I expect really need to be attended.

First and foremost, any session by Itzik Ben-Gan is going on my list, and I love the topic of the Beware Nondeterministic T-SQL Code section too. Honestly, I might know everything he is going to say in this presentation, but Itzik digs deep enough that there are usually an Aha Moment or twenty..

Really though, even sessions where you think you are an expert there is almost always the possibility of learning something new. Just last night, I reviewed a potential writer’s article on a subject I know quite well, but I still learned a few things! On the other hand, in Itzik’s case, it is possible that I might end up having no idea what he is talking about until I watch it a few times, so watching the recording won’t be all bad.

How to Maintain the Same Level of utilities in Cloud Deployments by Denny Cherry

As we all move to the cloud, even those of us who just write about SQL Server for the most part, this question is big in our minds. “How do I use the skills I have in the new world?” Plus, Denny is an entertaining speaker no matter what!

A Query Tuner’s Practical Guide to Statistics by Andy Yun.

Statistics are kind of a mystery to most people, even somewhat me at times. For the most part, we ignore them and let the engine do its thing. But sometimes… well a query is slow. Looking at the query plan, you see guesses about how many rows need to be processed and the actual number of rows processed is orders of magnitude greater. The common blame? Statistics.

Take knowledge that is useful and add to that the fact that Andy is a great speaker, and this one seems like a lock.

And so on

I could go on. I looked at the schedule quite a few times as part of the committee to choose sessions (I helped validate the sessions and schedules before starting here at Redgate) and there are so many great sessions no one could see them all, or even sit down and pick just a few to highlight. I didn’t even mention Paul Randal’s Performance Mythbusters session, Glenn Berry’s Index Tuning 101, Intelligent Data through Data Intelligence by Chris Unwin, SQL Titbits for the Inexperienced by Erland Sommarskog…any of the keynotes, precons, or so many others.

All this and these are just a handful (maybe two handfuls) of sessions that fit what I like best/ There are like 300 sessions plus to choose from all over the variety of topics the data platform provides.

See you soon!

 

The post The PASS Sessions I Am Most Excited For appeared first on Simple Talk.



from Simple Talk https://ift.tt/OjUQBlR
via