MySQL Performance Tuning

Many databases (and particularly many relational databases) rely on Structured Query Language (SQL), for handling data storage, data manipulation, and data retrieval. If developers want to create, update or delete data then they have always been able to do so easily with SQL statements. That said, we live in an age where the sheer amount of data being shunted around has grown and is still growing at alarming and overwhelming rates, and to compound this, workloads are always changing too, so while SQL statements are useful, there is an ongoing and pressing need for MySQL performance tuning. The swift and efficient movement and processing of data is crucial if we hope to deliver an excellent end-user experience while keeping costs as low as possible.

So, developers wanting to seek out and eliminate hold-ups and inadequate operations must turn to MySQL performance tuning tools which can help them with execution plans and remove any guesswork. MySQL performance may be important, but it isn’t necessarily an easy thing to do. In fact, there are a few aspects of the process which make it a difficult undertaking for developers. MySQL optimization requires sufficient technical prowess, i.e. enough knowledge and skill to comprehend and create a variety of execution plans, and that can be quite off-putting.

As if the fact that it’s tricky wasn’t enough, MySQL optimization takes time and energy. When you’re faced with a whole array of SQL statements to wade through, you face a problem with a built-in degree of uncertainty. Each example needs to be carefully considered during MySQL performance tuning. First, you need to decide which ones to amend and which to leave alone, then you need to work out what approach to take to MySQL tuning with each one that you do select because they’ll all need different approaches depending on what their function is. That’s why we are going to discuss several tips and techniques that will help you approach MySQL performance without getting distracted snowed under by the sheer weight of them.

The Benefits of MySQL Optimization

MySQL performance tuning is essential for keeping costs down. If you can use the right-sized server for the job, then you won’t be paying for more than you need, and if you can understand whether moving data storage or adding extra server capacity will lead to MySQL performance improvements then that helps efficiency too. MySQL tuning can be challenging, but it’s worth the time that it takes because an optimized database has greater responsiveness, delivers better performance, and offers better functionality.

MySQL Query Optimization Guidelines

Here are some useful tips for MySQL tuning. They are a great addition to your collection of best practices.

Make sure that the predicates in WHERE, JOIN, ORDER BY and GROUP BY clauses are all indexed. WebSphere Commerce points out that SQL performance can be improved significantly by predicate indexing because not doing so can result in table scans that culminate in locking and other difficulties. Which is why we highly recommend that you index all predicate columns for better MySQL optimization.

Keep functions out of predicates

The database won’t use an index if there’s a predefined function.

For instance:

  1. SELECT * FROM TABLEONE WHERE UPPER(COLONE)=’ABC’

The UPPER() function means that the database won’t look to the index on COLONE. If that function can’t be avoided in SQL, you’ll need to make a new function-based index or create custom columns in the database in order to experience improved MySQL performance.

Remove non-essential columns with the SELECT clause

Rather than use ‘SELECT *’, always specify columns for the SELECT clause, because unneeded columns add extra load on the database, hindering its performance and causing knock-on effects to the whole system.

Try not to use a wildcard (%) at the start of a predicate

The predicate LIKE ‘%abc’ will cause a full table scan, i.e.:

  1. SELECT * FROM TABLEONE WHERE COLONE LIKE ‘%ABC’

This kind of wildcard use can slow down performance significantly.

Use INNER JOIN, instead of OUTER JOIN where you can

Only use outer join where you absolutely need to. If you use it when you don’t need to then you’ll be putting the brakes on database performance due to slower execution of SQL statements and negative effects on MySQL optimization.

Use UNION AND DISTINCT only where needed

If you use the UNION and DISTINCT operators when other options are available you’ll be needlessly adding excessive sorting which slows down SQL performance. Try using UNION ALL instead for better MySQL performance.

You need to use ORDER BY in SQL for better sorting results

ORDER BY sorts the result-set into pre-determined statement columns. Although this is advantageous for database admins who want data to be sorted, it’s detrimental to MySQL performance. The reason for this is that in order to produce the final result set the query needs to sort the data first, which requires quite a convoluted and resource intensive SQL operation.

Don’t Use MySQL as a Queue

Queues can sneak up on you and slow down your database performance. For example, any time you set up a status for a specific item so that a ‘relevant process’ can gain access to it, you will be creating a queue without knowing it. This just adds pointless extra loading time to use the resource.

Queues are a problem because they cause your workload to be treated in an inefficient serial fashion rather than more efficient parallel and because they frequently lead to a table that contains work in progress along with data from jobs that have already been completed. This slows down the app and also hinders MySQL performance tuning.

The Four Fundamental Resources

A properly functioning database requires four fundamental resources. A CPU, hard drive, memory, and a network. Problems with any one of them will negatively affect the database, so it’s important to choose the right hardware and make sure that it’s all functioning properly. In practical terms, this means that if you’re going to invest in a powerful CPU then don’t try to cut corners with less memory or slower storage. A set up is only as good as its slowest component, and if all of them aren’t at parity then the result will be MySQL performance bottlenecks. Investing in more memory is probably the most cost-effective way of improving performance as it is inherently faster than disk-based storage. If all operations can be held in memory without resorting to disk usage, then processes will speed up considerably.

Pagination Queries

Applications that paginate tend to slow the server. By giving you a results page with a link to the next one these apps usually approach grouping and sorting in ways that don’t use indexes, using a LIMIT and offset function that places an extra burden on the server and then discards rows.

Adjusting the user interface itself will assist with optimization.  Instead of listing all pages in the results and linking to each page, you can just include a link to the next page. This also stops users wasting time on incorrect pages.

In terms of queries, rather than using LIMIT with offset, you can select one more row than you require, and when someone clicks the ‘next page’ link, you can set that last row as the start of the next set of results. For example, if the user looked at a page with rows 201 to 220, select row 221 as well; for the next page to be rendered, you would query the server for rows greater than or equal to 221, limit 21.

MySQL Optimization—Subqueries

In terms of subqueries, it’s better to use a join where you can, at least in current versions of MySQL.

The optimizer team is doing a lot of work on subqueries, so it may be that subsequent iterations of MySQL may come with extra subquery optimizations. It’s best to keep an eye on which MySQL optimizations end up in the each version, and what their effects will be. What I’m saying here is that my advice to err towards a join may not hold forever. Servers are getting more and more intelligent, and the instances where you will need to explain to them how to do something instead of what results to return are reducing.

Use Memcached for MySQL Caching

Memcached is a system that enables distributed memory caching, improving the speediness of websites that use big dynamic databases. It manages to do this by keeping database objects in Dynamic Memory to cut server load any time an outside data source asks for a read. A Memcached layer reduces the number of times a database issues a request.

Memcached stores each value (v) with a key (k), and then retrieves them without the need to parse the database queries for a much more streamlined process.

Conclusion

MySQL tuning  ( as well as tuning of MariaDB ) may be time-consuming and thought-provoking but it’s one of the hurdles that you need to take in your stride if you want to ensure that your users receive the best possible experience. Poor database performance could benefit from investing in the best hardware and making sure that it’s balanced, but even with the best CPUs, and the fastest memory and SSDs on the market, there is still an additional performance improvement to be had from taking the time to implement proper MySQL optimization. It can be a laborious task for developers, but the performance and enhancements and efficiency savings are well worth it. Keep these tips close to hand and refer to them often. They’re not all-encompassing, but they are a handy starting point for your journey into MySQL tuning.

Manage MySQL databases from the command line

Manage MySQL via command line

MySQL is one of the most popular relational database management systems. There are a variety of ways to manage a MySQL server – countless frontends including PhpMyAdmin, MySQL Workbench and even Adminer.
However, clearly the most efficient way to manage a MySQL server is directly through command line.

The mysql command-line tool comes with the MySQL DBMS and is a simple SQL shell solution with input line editing capabilities. It supports both non-interactive and interactive use.
In case of interactive use query results are presented in an ASCII-table format. In situation of non-interactive use the result is presented in tab-separated format. The output format can be changed using command options.

First you will need have to access your server via SSH in case of Linux. To connect to Windows server you need to use Remote Desktop. It is also possible to access your MySQL database via direct connection.

To connect to MySQL from the command line, follow these steps:

1. Once the command line is available, enter the following, where USERNAME is your database username:

mysql -u USERNAME -p

2. You’ll be prompted for a password, enter your password. You should then see the “mysql>” prompt.

3. You can see a list of databases by typing in this command:

show databases;

4. To perform database tasks you need to tell MySQL which database you want to use. To access a particular database, type the following command, where DBNAME is the database you want to access:

use DBNAME;

5. Once you’ve run this command you have access to your database. Next, you can execute queries, get a listing of MySQL tables and much more. By the way, typing “help” will get you a list of commands you can use, while typing “\q” will get you out of MySQL.

Adding new users and creating a new database

You can add users and databases when you are logged in as the root user in MySQL. To log in as root, and create new users and databases, follow these steps:

1. Log into MySQL with root privileges using the following command:

mysql -u root -p

2. You’ll be prompted for your root password, fill it in and press return to proceed.

3. Creating a database is straight forward, you just need the following command. When entering it, replace username with your user you would like to add. Do that with password too. This is the command you need to enter:

GRANT ALL PRIVILEGES ON *.* TO 'username'@'localhost' IDENTIFIED BY 'password';

4. Note that the command we listed above will give the new user you create blanket permissions. Instead of granting all privileges you can limit the user. One example is the SELECT permission. To limit the user to permissions as specified by SELECT only you need to enter the following command:

GRANT SELECT ON *.* TO 'username'@'localhost';

5. Type \q to exit MySQL so that you can log in with the user that you have just created.

6. It’s simple to log in to MySQL with your new user, just enter the login command we specified previously, typing the new user’s name instead of Username:

mysql -u username -p

7. The user we create have all privileges assigned to them, including the ability to create a database. Create a database with the following command, using the name of your database instead of DBNAME.

CREATE DATABASE dbname;

8. Want to start using this newly created database? Run the following command, again replacing DBNAME with the name of your database:

USE dbname;

9. MySQL now knows which database you want to work with. You can create a database table by running the following command, for example:

CREATE TABLE example ( id smallint unsigned not null auto_increment, name varchar(50) not null, constraint pk_example primary key (id) );
INSERT INTO example ( id, name ) VALUES ( null, 'Sample data' );

Scripting in SQL

You don’t need to run every single SQL command one command at a time, as in our previous example. Instead you can execute several commands all in one go by making use of a SQL script file.

This is how you use a SQL script file to create a new database, and to populate your new database:

1. Just like we did in the previous example we start by creating a new user for your database. These are the commands you need:

mysql -u root -p
GRANT ALL PRIVILEGES ON *.* TO 'username'@'localhost' IDENTIFIED BY 'password';
\q

2. Next, create a new file on your computer, and call it sql. You can use any text editor you like to use, as long as the file is stored in plain text.
3. Copy and paste the following into your text file:

CREATE DATABASE dbname;
USE dbname;
CREATE TABLE tablename ( id smallint unsigned not null auto_increment, name varchar(20) not null, constraint pk_example primary key (id) );
INSERT INTO tablename ( id, name ) VALUES ( null, 'Sample data' );

4. Before you save your file, change dbname into the name of your database and tablename into the name of the table you want to add to your new database.

Remember, you can modify this script to make as many tables as you like, and even to create multiple databases all in one go. Note that our example creates a very basic table: you might want to add more complex requirements by expanding on the script.

5. Save the changes to your file and close your text editor.

6. Processing your script is easy. Type the following command in, replacing username with the name of the user you created in the first step.

mysql -u username -p < example.sql

Note that MySQL will execute commands in a script one line at a time, in other words one statement at a time. In our example file, once MySQL has finished executing the entire file you will notice that a new database and a new table is created. Finally, the table will contain any data that you have specified in the INSERT statement.

Deleting a specific table, or an entire database

Performing a table deletion is not hard. Just type this command once you are logged into MySQL, using the name of the table you want to delete instead of tablename.

DROP TABLE tablename;

Note that to execute this command you need to have already selected which database you are working on via the use command.

Deleting a complete database is simple too. Just execute this command at the MySQL prompt, again replacing dbname with the name of the database you want to remove:

DROP DATABASE dbname;

Beware: MySQL will immediately remove the database when you submit that command, it does not ask for any type of confirmation: the database is permanently removed, including all of its contents.

Deleting a database user

You can view a full list of every database user by running the following command:

SELECT user FROM mysql.user GROUP BY user;

Deleting a single user is simple, just run the following command, but insert the name of the user you want to delete in the place of username:

DELETE FROM mysql.user WHERE user = 'username';