Professional Documents
Culture Documents
Unit 2
Data Types
24
Unit 3
Data Definition Language (DDL)
52
Unit 4
Data Manipulation Language (DML)
90
Unit 5
Advanced Data Manipulation Language
114
Unit 6
Subqueries
133
Unit 7
Operators and Functions
153
Unit 8
Advanced Functions
183
Unit 9
Transaction Management
214
Unit 10
Stored Procedures
229
Unit 11
Control Statement
242
Unit 12
User Account Management
253
Unit 13
General Security Issues
281
Unit 14
Log Files
294
References
311
Prof. S. Kannan
Director & Dean (in-charge)
Directorate of Distance Education
Sikkim Manipal University of Health, Medical & Technological Sciences (SMU DDE)
Board of Studies
Dr. U. B. Pavanaja (Chairman)
General Manager Academics
Manipal Universal Learning Pvt. Ltd.
Bangalore.
Prof. Bhushan Patwardhan
Chief Academics
Manipal Education
Bangalore.
Dr. Harishchandra Hebbar
Director
Manipal Centre for Info. Sciences,
Bangalore
Dr. N. V. Subba Reddy
HOD-CSE
Manipal Institute of Technology, Manipal
Dr. Ashok Hegde
Vice President
MindTree Consulting Ltd., Bangalore
Dr. Ramprasad Varadachar
Director, Computer Studies
Dayanand Sagar College of Engg. Bangalore.
Content Preparation Team
Content Writing
Mr. Nirmal Kumar Nigam
Assistant Professor, HoP IT
Sikkim Manipal University DDE
Manipal.
Instructional Design
Mr. Kulwinder Pal
Senior Lecturer (Education)
Sikkim Manipal University DDE, Manipal
Content Editing
Mr. Vinayak G. Pai
Assistant Professor, Dept. of IT
Sikkim Manipal University DDE
Manipal.
SUBJECT INTRODUCTION
RDBMS with MySQL is a 4 credit subject in third semester of BSc IT
program.
MySQL, the most popular Open Source SQL database management
system, is developed, distributed, and supported by MySQL AB.
Open Source means that it is possible for anyone to use and modify the
software. Anybody can download the MySQL software from the Internet and
use it without paying anything. If you wish, you may study the source code
and change it to suit your needs. The MySQL software uses the GPL (GNU
General Public License), http://www.fsf.org/licenses/, to define what you
may and may not do with the software in different situations.
The MySQL Database Server is very fast, reliable, and easy to use. MySQL
Server works in client/server or embedded systems.
MySQL is named after co-founder Monty Widenius's daughter, My.
In this book on MySQL, we would lay the emphasis on the applications of
MySQL in solving the database related problems and also deal with the
coverage of every aspect related to MySQL.
The SLM of this subject is divided into 14 units, the overview of which is
given below.
Unit 1 Basics of MySQL:
This unit introduces the user to the basic theoretical concepts and features
behind MySQL.
Unit 2 MySQL Data Types:
This unit introduces the theoretical and practical aspects of dealing with
various data types of MySQL database, their syntax, usage and applications
in the real world.
Unit 3 Data Definition Language (DDL):
This unit starts with an introduction of Data Definition Language Statements
used in MySQL. It describes the syntaxes of creating and modifying
Databases, Indexes and Tables.
Unit 1
Unit 1
Basics of MySQL
Structure
1.1 Introduction
Objectives
1.2 Features of MySQL
1.3 Top 10 Reasons to use MySQL
1.4 MySQL Development Roadmap
1.5 Connecting to and Disconnecting from the Server
1.6 Accessing and Creating Databases and Tables
1.7 Loading Data
1.8 Summary
1.9 Terminal Questions
1.10 Answers
1.1 Introduction
MySQL, the most popular Open Source SQL database management
system, is developed, distributed, and supported by MySQL AB. MySQL AB
is a commercial company, founded by the MySQL developers. It is a second
generation Open Source company that unites Open Source values and
methodology with a successful business model.
The MySQL Web site (http://www.mysql.com/) provides the latest
information about MySQL software and MySQL AB.
MySQL is a Database Management System
A database is a structured collection of data. It may be anything from a
simple shopping list to a picture gallery or the vast amounts of
information in a corporate network. To add, access, and process data
stored in a computer database, you need a database management
system such as MySQL Server.
MySQL is a Relational Database Management System
A Relational Database stores data in separate tables rather than putting
all the data in one big storeroom. This adds speed and flexibility.
MySQL software is Open Source
Open Source means that it is possible for anyone to use and modify the
software. Anybody can download the MySQL software from the Internet
and use it without paying anything. If you wish, you may study the
Sikkim Manipal University
Page No. 1
Unit 1
Page No. 2
Unit 1
Page No. 3
Unit 1
Full support for SQL GROUP BY and ORDER BY clauses. Support for
group functions (COUNT(), COUNT(DISTINCT ...), AVG(), STD(),
SUM(), MAX(), MIN(), and GROUP_CONCAT()).
Support for LEFT OUTER JOIN and RIGHT OUTER JOIN with both
standard SQL and ODBC syntax.
Support for aliases on tables and columns as required by standard SQL.
DELETE, INSERT, REPLACE, and UPDATE return the number of rows
that were changed (affected). It is possible to return the number of rows
matched instead by setting a flag when connecting to the server.
The MySQL-specific SHOW statement can be used to retrieve
information about databases, storage engines, tables, and indexes.
MySQL 5.0 adds support for the INFORMATION_SCHEMA database,
implemented according to standard SQL.
The EXPLAIN statement can be used to determine how the optimizer
resolves a query.
Function names do not clash with table or column names. For example,
ABS is a valid column name. The only restriction is that for a function
call, no spaces are allowed between the function name and the ( that
follows it.
You can refer to tables from different databases in the same statement.
Security:
A privilege and password system that is very flexible and secure, and
that allows host-based verification.
Passwords are secure because all password traffic is encrypted when
you connect to a server.
Scalability and Limits:
Handles large databases. We use MySQL Server with databases that
contain 50 million records. We also know of users who use MySQL
Server with 60,000 tables and about 5,000,000,000 rows.
Up to 64 indexes per table are allowed (32 before MySQL 4.1.2). Each
index may consist of 1 to 16 columns or parts of columns. The maximum
Sikkim Manipal University
Page No. 4
Unit 1
index width is 1000 bytes (767 for InnoDB); before MySQL 4.1.2, the
limit is 500 bytes. An index may use a prefix of a column for CHAR,
VARCHAR, BLOB, or TEXT column types.
Connectivity:
Clients can connect to MySQL Server using several protocols:
Clients can connect using TCP/IP sockets on any platform.
On Windows systems in the NT family (NT, 2000, XP, 2003, or
Vista), clients can connect using named pipes if the server is started
with the enable-named-pipe option. In MySQL 4.1 and higher,
Windows servers also support shared-memory connections if started
with the shared-memory option. Clients can connect through
shared memory by using the protocol=memory option.
On Unix systems, clients can connect using Unix domain socket
files.
MySQL client programs can be written in many languages. A client
library written in C is available for clients written in C or C++, or for any
language that provides C bindings.
APIs for C, C++, Eiffel, Java, Perl, PHP, Python, Ruby, and Tcl are
available, allowing MySQL clients to be written in many languages.
The Connector/ODBC (MyODBC) interface provides MySQL support for
client programs that use ODBC (Open Database Connectivity)
connections. For example, you can use MS Access to connect to your
MySQL server. Clients can be run on Windows or Unix. MyODBC
source is available. All ODBC 2.5 functions are supported, as are many
others.
The Connector/J interface provides MySQL support for Java client
programs that use JDBC connections. Clients can be run on Windows or
Unix. Connector/J source is available.
MySQL Connector/NET enables developers to easily create .NET
applications that require secure, high-performance data connectivity with
MySQL. It implements the required ADO.NET interfaces and integrates
into ADO.NET aware tools. Developers can build applications using their
choice of .NET languages. MySQL Connector/NET is a fully managed
ADO.NET driver written in 100% pure C#.
Page No. 5
Unit 1
Localization:
The server can provide error messages to clients in many languages.
Full support for several different character sets, including latin1
(cp1252), german, big5, ujis, and more. For example, the Scandinavian
characters , and are allowed in table and column names.
Unicode support is available as of MySQL 4.1.
All data is saved in the chosen character set.
Sorting and comparisons are done according to the chosen character
set and collation (using latin1 and Swedish collation by default). It is
possible to change this when the MySQL server is started. To see an
example of very advanced sorting, look at the Czech sorting code.
MySQL Server supports many different character sets that can be
specified at compile time and runtime.
As of MySQL 4.1, the server time zone can be changed dynamically,
and individual clients can specify their own time zone.
Clients and Tools:
MySQL AB provides several client and utility programs. These include
both command-line programs such as mysqldump and mysqladmin,
and graphical programs such as MySQL Administrator and MySQL
Query Browser.
MySQL Server has built-in support for SQL statements to check,
optimize, and repair tables. These statements are available from the
command line through the mysqlcheck client. MySQL also includes
myisamchk, a very fast command-line utility for performing these
operations on MyISAM tables.
MySQL programs can be invoked with the --help or -? option to obtain
online assistance.
Self Assessment Questions
1. MySQL is a _______________ .
2. The _______ is a privileged and Password system that is very flexible
and secure, and that allows host-based verification.
Page No. 6
Unit 1
Page No. 7
Unit 1
Page No. 8
Unit 1
Page No. 9
Unit 1
MySQL Series
4.0
4.1
4.1 (for the MyISAM storage
engine)
5.0
5.0
5.0
5.0
5.0 and 5.1
5.1
5.1
5.1
5.1
5.1
5.1
6.x (implemented in 3.23 for
InnoDB)
Page No. 10
Unit 1
Page No. 11
Unit 1
and time. Conceptually, this is similar to the idea of the Unix crontab
(also known as a cron job) or the Windows Task Scheduler.
Server Log Tables. Before MySQL 5.1, the server writes general query
log and slow query log entries to log files. As of MySQL 5.1, the server's
logging capabilities for these logs are more flexible. Log entries can be
written to log files (as before) or to the general_log and slow_log tables
in the mysql database. If logging is enabled, either or both destinations
can be selected. The --log-output option controls the destination or
destinations of log output.
Upgrade Program. The mysql_upgrade program (available as of
MySQL 5.1.7) checks all existing tables for incompatibilities with the
current version of MySQL Server and repairs them if necessary. This
program should be run for each MySQL upgrade.
MySQL Cluster. MySQL Cluster is now released as a separate
product, based on MySQL 5.1 but with the addition of the
NDBCLUSTER storage engine. Clustering support is longer available in
mainline MySQL 5.1 releases. MySQL Cluster releases are identified by
a 3-part NDB version number; currently, the MySQL Cluster NDB 6.2
and MySQL Cluster NDB 6.3 release series are available for production
use.
Self Assessment Questions
3. The number of indexes allowed per table in MySQL are _______.
Page No. 12
Unit 1
host and user represent the host name where your MySQL server is
running and the user name of your MySQL account. Substitute appropriate
values for your setup. The ******** represents your password; enter it when
mysql displays the Enter password: prompt.
If that works, you should see some introductory information followed by a
mysql> prompt:
shell> mysql -h host -u user -p
Enter password: ********
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 25338 to server version: 5.1.30-standard
Type 'help;' or '\h' for help. Type '\c' to clear the buffer.
mysql>
The mysql> prompt tells you that mysql is ready for you to enter
commands.
If you are logging in on the same machine that MySQL is running on, you
can omit the host, and simply use the following:
shell> mysql -u user -p
If, when you attempt to log in, you get an error message such as ERROR
2002 (HY000): Can't connect to local MySQL server through socket
'/tmp/mysql.sock' (2), it means that that MySQL server daemon (Unix) or
service (Windows) is not running. Consult the administrator
Some MySQL installations allow users to connect as the anonymous
(unnamed) user to the server running on the local host. If this is the case on
your machine, you should be able to connect to that server by invoking
mysql without any options:
shell> mysql
After you have connected successfully, you can disconnect any time by
typing QUIT (or \q) at the mysql> prompt:
mysql> QUIT
Bye
Page No. 13
Unit 1
Most examples in the following sections assume that you are connected to
the server. They indicate this by the mysql> prompt.
Page No. 14
Unit 1
The following screen shot shows the appearance of mysql> prompt after the
user types in the correct password.
The following screen shot shows an error message wherein the user tries to
access a table without using any database; i.e. as s first step, the user must
select a database already present by using the USE <DATABASE-NAME>
command and then access the tables, views or any other data structures
within the database.
Other wise the error message Error 1046: No database Selected would be
generated by the database server.
Page No. 15
Unit 1
The screen shot below shows the command Create Database <DatabaseName> used to create a database by the user.
After successful execution of the above command, the user can type in the
Show Databases command to find out the successful creation of the
database created by him / her. Then the user can start creating his data
objects like tables inside the database.
Page No. 16
Unit 1
The following screenshot shows how the user can select his own database
created in the previous step and start creating tables using Create Table
command. The DESC command can be used to show the metadata
regarding the table created.
Page No. 17
Unit 1
Step 1: Opening a Notepad file from the Start menu of the Windows OS
Step 2: The data should be tab separated and the number of columns
correpsond to the number of columns of the base table. The data typed in
the notepad file should correspond to the data type set in the underlying
table.
Page No. 18
Unit 1
Page No. 19
Unit 1
Step 4: Creation of the underlying base table to which the data values
should be populated from the notepad file created in the C drive of your hard
disk.
In this process we use the LOAD DATA INFILE C:\FileName.txt
command which enables us to load the data into the specified database
table.
Page No. 20
Unit 1
1.8 Summary
This unit has covered the following topics:
1. Features of MySQL: Some of the major features discussed here are
Internals and Portability, Data Type support, Security Issues, etc.
Page No. 21
Unit 1
2. Top Reasons for using MySQL: The top 10 reasons for using MySQL
by majority of the user community are discussed here like Scalability &
Flexibility, High Performance, Robust Transactional support, Web and
Data Warehouse strengths, Stronger data protection, etc.
3. MySQL Development Roadmap: This topic provides an overview
about features that have been implemented in previous series and that
are new in this current release series (5.1), and an overview about
upcoming additions or changes in the next release series (6.0).
4. Connecting to and Disconnecting from the Server: This section
discusses about the syntax used to connect to the database server and
disconnect from the database server using command line interface.
5. Accessing and Creating Databases and Tables: This topic discusses
various methods and functions used to access and create databases
and tables within the MySQL server database.
6. Loading Data: This topic discusses about the methods of loading data
into the MySQL database tables from external files such as notepad
data.
1.10 Answers
Self Assessment Questions
1. Relational Database Management System
2. Security
3. 64
Terminal Questions
1. 1. Portability
2. Fast Thread based Memory Allocation System
3. Full operator and Functional Support
4. Support for Group By and Order By Clauses
(Refer to section 1.2)
Sikkim Manipal University
Page No. 22
Unit 1
Page No. 23
Unit 2
Unit 2
Data Types
Structure
2.1
Introduction
Objectives
2.2
Numeric Types
2.3
String Types
The CHAR and VARCHAR Types
The BLOB and TEXT Types
The ENUM Type
The SET Type
2.4
Date and Time Types
The DATETIME, DATE, and TIMESTAMP Types
The TIME Type
The YEAR Type
Y2K Issues and Date Types
2.5
Column Type Storage Requirements
2.6
Choosing the Right Type for a Column
2.7
Using Column Types from Other Database Engines
2.8
Summary
2.9
Terminal Questions
2.10 Answers
2.1 Introduction
In this unit the information regarding the data types uses in MySQL is
presented. An exhaustive discussion regarding the Numeric data types is
made with suitable examples is given. The String data type is used to
manipulate strings and characters; In this regards the data types such as
CHAR, VARCHAR, BLOB, and so on are discussed.
The Date and Time data types are also discussed.
In addition, the column storage requirements of different data types are
mentioned, so that the user can choose the right column types for his
database.
Page No. 24
Unit 2
Objectives
After studying this unit, you should be able to:
describe the importance of data types
apply the Numeric, String and Date & Time data types
describe the column storage requirements in MySQL
explain how to choose a right column type for individual columns
describe how to use column types from other database engines
Page No. 25
Unit 2
Page No. 26
Unit 2
However, the values TRUE and FALSE are merely aliases for 1 and 0,
respectively, as shown here:
The last two statements display the results shown because 2 is equal to
neither 1 nor 0.
We intend to implement full boolean type handling, in accordance with
standard SQL, in a future MySQL release.
SMALLINT[(M)] [UNSIGNED] [ZEROFILL]: A small integer. The signed
range is -32768 to 32767. The unsigned range is 0 to 65535.
MEDIUMINT[(M)] [UNSIGNED] [ZEROFILL]: A medium-sized integer.
The signed range is -8388608 to 8388607. The unsigned range is 0 to
16777215.
INT[(M)] [UNSIGNED] [ZEROFILL]: A normal-size integer. The signed
range is -2147483648 to 2147483647. The unsigned range is 0 to
4294967295.
INTEGER[(M)] [UNSIGNED] [ZEROFILL]: This type is a synonym for
INT.
Sikkim Manipal University
Page No. 27
Unit 2
Page No. 28
Unit 2
Note: Using FLOAT might give you some unexpected problems because all
calculations in MySQL are done with double precision.
DOUBLE [(M,D)] [UNSIGNED] [ZEROFILL]:
A normal - size (double-precision) floating-point number.
Allowable values are -1.7976931348623157E+308 to 2.2250738585072014E-308, 0, and 2.2250738585072014E-308 to
1.7976931348623157E+308.
These are the theoretical limits, based on the IEEE standard. The actual
range might be slightly smaller depending on your hardware or operating
system.
M is the total number of digits and D is the number of digits following the
decimal point. If M and D are omitted, values are stored to the limits allowed
by the hardware. A double-precision floating-point number is accurate to
approximately 15 decimal places.
UNSIGNED, if specified, disallows negative values.
DOUBLE PRECISION [(M,D)] [UNSIGNED] [ZEROFILL], REAL[(M,D)]
[UNSIGNED] [ZEROFILL]: These types are synonyms for DOUBLE.
Exception: If the REAL_AS_FLOAT SQL mode is enabled, REAL is a
synonym for FLOAT rather than DOUBLE.
FLOAT(p) [UNSIGNED] [ZEROFILL]: A floating - point number. p
represents the precision in bits, but MySQL uses this value only to
determine whether to use FLOAT or DOUBLE for the resulting data type. If
p is from 0 to 24, the data type becomes FLOAT with no M or D values. If p
is from 25 to 53, the data type becomes DOUBLE with no M or D values.
The range of the resulting column is the same as for the single-precision
FLOAT or double-precision DOUBLE data types described earlier in this
section.
Note: FLOAT(p) syntax is provided for ODBC compatibility.
DECIMAL [(M[,D])] [UNSIGNED] [ZEROFILL]: A packed exact fixed-point
number. M is the total number of digits (the precision) and D is the number
of digits after the decimal point (the scale). The decimal point and (for
negative numbers) the - sign are not counted in M. If D is 0, values have
no decimal point or fractional part. The maximum number of digits (M) for
Sikkim Manipal University
Page No. 29
Unit 2
Page No. 30
Unit 2
If strict SQL mode is not enabled and you assign a value to a CHAR or
VARCHAR column that exceeds the column's maximum length, the value is
truncated to fit and a warning is generated. For truncation of non-space
characters, you can cause an error to occur (rather than a warning) and
suppress insertion of the value by using strict SQL mode.
For VARCHAR columns, trailing spaces in excess of the column length are
truncated prior to insertion and a warning is generated, regardless of the
SQL mode in use. For CHAR columns, truncation of excess trailing spaces
from inserted values is performed silently regardless of the SQL mode.
VARCHAR values are not padded when they are stored. Trailing spaces are
retained when values are stored and retrieved, in conformance with
standard SQL.
The following table illustrates the differences between CHAR and
VARCHAR by showing the result of storing various string values into
CHAR(4) and VARCHAR(4) columns (assuming that the column uses a
single-byte character set such as latin1):
Table 2.1: Differences between Char and Varchar Variables
Value
CHAR(4)
ab
Storage
Required
VARCHAR(4)
Storage
Required
4 bytes
1 bytes
ab
4 bytes
ab
3 bytes
abcd
abcd
4 bytes
abcd
5 bytes
abcdefgh
abcd
4 bytes
abcd
5 bytes
The values shown as stored in the last row of the table apply only when not
using strict mode; if MySQL is running in strict mode, values that exceed the
column length are not stored, and an error results.
If a given value is stored into the CHAR(4) and VARCHAR(4) columns, the
values retrieved from the columns are not always the same because trailing
spaces are removed from CHAR columns upon retrieval. The following
example illustrates this difference:
Page No. 31
Unit 2
This is true for all MySQL versions, and that it is not affected by the server
SQL mode.
Page No. 32
Unit 2
For those cases where trailing pad characters are stripped or comparisons
ignore them, if a column has an index that requires unique values, inserting
into the column values that differ only in number of trailing pad characters
will result in a duplicate-key error. For example, if a table contains 'a', an
attempt to store 'a' causes a duplicate-key error.
2.3.2 The BLOB and TEXT Types
A BLOB is a binary large object that can hold a variable amount of data. The
four BLOB types are TINYBLOB, BLOB, MEDIUMBLOB, and LONGBLOB.
These differ only in the maximum length of the values they can hold. The
four TEXT types are TINYTEXT, TEXT, MEDIUMTEXT, and LONGTEXT.
These correspond to the four BLOB types and have the same maximum
lengths and storage requirements.
BLOB columns are treated as binary strings (byte strings). TEXT columns
are treated as non-binary strings (character strings). BLOB columns have no
character set, and sorting and comparison are based on the numeric values
of the bytes in column values. TEXT columns have a character set, and
values are sorted and compared based on the collation of the character set.
If strict SQL mode is not enabled and you assign a value to a BLOB or
TEXT column that exceeds the column's maximum length, the value is
truncated to fit and a warning is generated. For truncation of non-space
characters, you can cause an error to occur (rather than a warning) and
suppress insertion of the value by using strict SQL mode. Beginning with
MySQL 5.1.24, truncation of excess trailing spaces from values to be
inserted into TEXT columns always generates a warning, regardless of the
SQL mode. (Bug#30059)
If a TEXT column is indexed, index entry comparisons are space-padded at
the end. This means that, if the index requires unique values, duplicate-key
errors will occur for values that differ only in the number of trailing spaces.
For example, if a table contains 'a', an attempt to store 'a ' causes a
duplicate-key error. This is not true for BLOB columns.
In most respects, you can regard a BLOB column as a VARBINARY column
that can be as large as you like. Similarly, you can regard a TEXT column
as a VARCHAR column. BLOB and TEXT differ from VARBINARY and
VARCHAR in the following ways:
Sikkim Manipal University
Page No. 33
Unit 2
For indexes on BLOB and TEXT columns, you must specify an index
prefix length. For CHAR and VARCHAR, a prefix length is optional.
BLOB and TEXT columns cannot have DEFAULT values.
LONG and LONG VARCHAR map to the MEDIUMTEXT data type. This is a
compatibility feature. If you use the BINARY attribute with a TEXT data type,
the column is assigned the binary collation of the column character set.
MySQL Connector/ODBC defines BLOB values as LONGVARBINARY and
TEXT values as LONGVARCHAR.
Because BLOB and TEXT values can be extremely long, you might
encounter some constraints in using them:
Only the first max_sort_length bytes of the column are used when
sorting. The default value of max_sort_length is 1024. This value can be
changed using the --max_sort_length=N option when starting the mysqld
server.
You can make more bytes significant in sorting or grouping by increasing
the value of max_sort_length at runtime. Any client can change the value of
its session max_sort_length variable:
mysql> SET max_sort_length = 2000;
mysql> SELECT id, comment FROM t
-> ORDER BY comment;
The maximum size of a BLOB or TEXT object is determined by its type, but
the largest value you actually can transmit between the client and server is
determined by the amount of available memory and the size of the
Sikkim Manipal University
Page No. 34
Unit 2
Page No. 35
Unit 2
This makes the SET data type a good choice for multiple-choice selections
as shown in the example given below:
Note: As with ENUM type, any attempt to use a value that is not part of the
predefined set will cause MySQL to print an error message instead as
shown in the example below:
Page No. 36
Unit 2
an illegal value that MySQL cannot represent. The TIMESTAMP type has
special automatic updating behavior, described later on.
MySQL gives warnings or errors if you try to insert an illegal date. By setting
the SQL mode to the appropriate value, you can specify more exactly what
kind of dates you want MySQL to support.
You can get MySQL to accept certain dates, such as '2009-11-31', by using
the ALLOW_INVALID_DATES SQL mode. This is useful when you want to
store a possibly wrong value which the user has specified (for example, in
a web form) in the database for future processing.
Under this mode, MySQL verifies only that the month is in the range from 0
to 12 and that the day is in the range from 0 to 31. These ranges are defined
to include zero because MySQL allows you to store dates where the day or
month and day are zero in a DATE or DATETIME column.
This is extremely useful for applications that need to store a birthdate for
which you do not know the exact date. In this case, you simply store the
date as '2009-00-00' or '2009-01-00'.
If you store dates such as these, you should not expect to get correct results
for functions such as DATE_SUB() or DATE_ADD() that require complete
dates. (If you do not want to allow zero in dates, you can use the
NO_ZERO_IN_DATE SQL mode).
Prior to MySQL 5.1.18, when DATE values are compared with DATETIME
values, the time portion of the DATETIME value is ignored, or the
comparison could be performed as a string compare.
Starting from MySQL 5.1.18, a DATE value is coerced to the DATETIME
type by adding the time portion as '00:00:00'. To mimic the old behavior, use
the CAST() function to cause the comparison operands to be treated as
previously.
For example:
date_col = CAST(NOW() AS DATE);
MySQL also allows you to store '0000-00-00' as a dummy date (if you are
not using the NO_ZERO_DATE SQL mode). This is in some cases more
convenient (and uses less data and index space) than using NULL values.
Sikkim Manipal University
Page No. 37
Unit 2
Following are the general considerations to keep in mind when working with
date and time types:
MySQL retrieves values for a given date or time type in a standard
output format, but it attempts to interpret a variety of formats for input
values that you supply (for example, when you specify a value to be
assigned to or compared to a date or time type). Only the formats
described in the following sections are supported. It is expected that you
supply legal values. Unpredictable results may occur if you use values in
other formats.
Dates containing two-digit year values are ambiguous because the
century is unknown. MySQL interprets two-digit year values using the
following rules:
o Year values in the range 70-99 are converted to 1970-1999.
o Year values in the range 00-69 are converted to 2000-2069.
Although MySQL tries to interpret values in several formats, dates
always must be given in year-month-day order (for example, '98-0904'), rather than in the month-day-year or day-month-year orders
commonly used elsewhere (for example, '09-04-98', '04-09-98').
MySQL automatically converts a date or time type value to a number if
the value is used in a numeric context and vice versa.
By default, when MySQL encounters a value for a date or time type that
is out of range or otherwise illegal for the type (as described at the
beginning of this section), it converts the value to the zero value for
that type. The exception is that out-of-range TIME values are clipped to
the appropriate endpoint of the TIME range.
The following table shows the format of the zero value for each type.
Note that the use of these values produces warnings if the
NO_ZERO_DATE SQL mode is enabled.
Data Type
Zero Value
DATETIME
'0000-00-00 00:00:00'
DATE
'0000-00-00'
TIMESTAMP
'0000-00-00 00:00:00'
TIME
'00:00:00'
YEAR
0000
Page No. 38
Unit 2
The table below lists the various date and time data types, together with
their allowed ranges and formats:
Table 2.2: Date and Time Data Types
Type
Size
(bytes)
Range
Format
Used For
DATE
1000-01-01 to
9999-12-31
YYYY-MM-DD
Date Values
TIME
-838:59:59 to
838:59:59
HH:MM:SS
Time values or
durations
YEAR
1901 to 2155
YYYY
Year Values
DATETIME
1000-01-01
00:00:00 to
9999-12-31
23:59:59
YYYY-MM-DD
HH:MM:SS
Combined date
and time values
TIMESTAMP
1970-01-01
00:00:01 to
2038-01-09
03:14:07
YYYYMMDDHHM
MSS
Combined date
and time values,
timestamps
Page No. 39
Unit 2
You can specify DATETIME, DATE, and TIMESTAMP values using any of a
common set of formats:
As a string in either 'YYYY-MM-DD HH:MM:SS' or 'YY-MM-DD
HH:MM:SS' format. A relaxed syntax is allowed: Any punctuation
character may be used as the delimiter between date parts or time parts.
For example, '98-12-31 11:30:45', '98.12.31 11+30+45', '98/12/31
11*30*45', and '98@12@31 11^30^45' are equivalent.
As a string in either 'YYYY-MM-DD' or 'YY-MM-DD' format. A relaxed
syntax is allowed here, too. For example, '98-12-31', '98.12.31',
'98/12/31', and '98@12@31' are equivalent.
As a string with no delimiters in either 'YYYYMMDDHHMMSS' or
'YYMMDDHHMMSS' format, provided that the string makes sense as a
date. For example, '20070523091528' and '070523091528' are
interpreted as '2007-05-23 09:15:28', but '071122129015' is illegal (it has
a nonsensical minute part) and becomes '0000-00-00 00:00:00'.
As a string with no delimiters in either 'YYYYMMDD' or 'YYMMDD'
format, provided that the string makes sense as a date. For example,
'20070523' and '070523' are interpreted as '2007-05-23', but '071332' is
illegal (it has nonsensical month and day parts) and becomes '0000-0000'.
As a number in either YYYYMMDDHHMMSS or YYMMDDHHMMSS
format, provided that the number makes sense as a date. For example,
19830905132800 and 830905132800 are interpreted as '1983-09-05
13:28:00'.
As a number in either YYYYMMDD or YYMMDD format, provided that
the number makes sense as a date. For example, 19830905 and
830905 are interpreted as '1983-09-05'.
As the result of a function that returns a value that is acceptable in a
DATETIME, DATE, or TIMESTAMP context, such as NOW() or
CURRENT_DATE.
A microseconds part is allowable in temporal values in some contexts, such
as in literal values, and in the arguments to or return values from some
Page No. 40
Unit 2
Page No. 41
Unit 2
DATETIME.) By default, the current time zone for each connection is the
server's time. The time zone can be set on a per-connection basis.
As long as the time zone setting remains constant, you get back the same
value you store. If you store a TIMESTAMP value, and then change the time
zone and retrieve the value, the retrieved value is different from the value
you stored. This occurs because the same time zone was not used for
conversion in both directions.
The current time zone is available as the value of the time_zone system
variable.
The TIMESTAMP data type offers automatic initialization and updating. You
can choose whether to use these properties and which column should have
them
2.4.2 The TIME Type
MySQL retrieves and displays TIME values in 'HH:MM:SS' format (or
'HHH:MM:SS' format for large hours values). TIME values may range from '838:59:59' to '838:59:59'. The hours part may be so large because the
TIME type can be used not only to represent a time of day (which must be
less than 24 hours), but also elapsed time or a time interval between two
events (which may be much greater than 24 hours, or even negative).
You can specify TIME values in a variety of formats:
As a string in 'D HH:MM:SS.fraction' format. You can also use one of
the following relaxed syntaxes: 'HH:MM:SS.fraction', 'HH:MM:SS',
'HH:MM', 'D HH:MM:SS', 'D HH:MM', 'D HH', or 'SS'. Here D represents
days and can have a value from 0 to 34. Note that MySQL does not
store the fraction part.
As a string with no delimiters in 'HHMMSS' format, provided that it
makes sense as a time. For example, '101112' is understood as
'10:11:12', but '109712' is illegal (it has a nonsensical minute part) and
becomes '00:00:00'.
As a number in HHMMSS format, provided that it makes sense as a
time. For example, 101112 is understood as '10:11:12'. The following
alternative formats are also understood: SS, MMSS, HHMMSS,
HHMMSS.fraction. Note that MySQL does not store the fraction part.
Page No. 42
Unit 2
Page No. 43
Unit 2
Page No. 44
Unit 2
Page No. 45
Unit 2
1. Numeric Types:
Table 2.3: Numeric Data Types
Column Type
Storage Required
(in bytes)
TINYINT
SMALLINT
MEDIUMINT
INT
INTEGER
BIGINT
FLOAT(X)
FLOAT
DOUBLE
DOUBLE PRECISION
REAL
DECIMAL(M,D), NUMERIC(M,D)
DATE
DATETIME
TIMESTAMP
TIME
YEAR
Page No. 46
Unit 2
String Types:
Table 2.4: String Data Type Storage Requirements
Column Type
CHAR(M)
VARCHAR(M)
TINYBLOB, TINYTEXT
BLOB, TEXT
MEDIUMBLOB, MEDIUMTEXT
LONGBLOB, LONGTEXT
ENUM('value1','value2',...)
SET('value1','value2',...)
Storage Required
(in bytes)
M bytes, 0 <= M <= 255
L+1 bytes, where L <= M and 0 <= M <=
255
L+1 bytes, where L < 2^8
L+2 bytes, where L < 2^16
L+3 bytes, where L < 2^24
L+4 bytes, where L < 2^32
1 or 2 bytes, depending on the number of
enumeration
values (65,535 values maximum)
1, 2, 3, 4, or 8 bytes, depending on the
number of set members
(64 members maximum)
VARCHAR and the BLOB and TEXT types are variable-length types, for
which the storage requirements depend on the actual length of column
values (represented by L in the preceding table), rather than on the types
maximum possible size.
For example, a VARCHAR(10) column can hold a string with a maximum
length of 10 characters. The actual storage required is the length of the
string (L), plus 1 byte to record the length of the string. For the string 'abcd',
L is 4 and the storage requirement is 5 bytes.
The BLOB and TEXT types require 1, 2, 3, or 4 bytes to record the length of
the column
value, depending on the maximum possible length of the type.
The size of an ENUM object is determined by the number of different
enumeration values. One byte is used for enumerations with up to 255
possible values. Two bytes are used for enumerations with up to 65535
values.
Page No. 47
Unit 2
MySQL Type
BINARY(NUM)
CHAR(NUM) BINARY
CHAR VARYING(NUM)
VARCHAR(NUM)
FLOAT4
FLOAT
FLOAT8
DOUBLE
INT1
TINYINT
Page No. 48
Unit 2
INT2
SMALLINT
INT3
MEDIUMINT
INT4
INT
INT8
BIGINT
LONG VARBINARY
MEDIUMBLOB
LONG VARCHAR
MEDIUMTEXT
LONG
MIDDLEINT
MEDIUMINT
VARBINARY(NUM)
VARCHAR(NUM) BINARY
Column type mapping occurs at table creation time. If you create a table
with types used by other vendors and then issue a DESCRIBE tbl_name
statement, MySQL reports the table structure using the equivalent MySQL
types.
2.8 Summary
This unit covers the following topics with respect to data types in MySQL:
1. Numeric Types: This topic discusses various Numeric data types like
Bit, Float, Integer, and so on and their various syntaxes available for
usage in numerical calculations of SQL statements.
2. String Types: This topic discusses the character data types and their
variants like CHAR and VARCHAR, String data types and their
variations.
3. Date and Time Types: This topic discusses the different variants of
Date, DateTime, and TimeStamp data types used in MySQL along with
their syntaxes and the context of usage.
4. Column Type Storage Requirements: This topic discusses the storage
requirements for each of the column types so that the user can judge the
optimum data types for his database for saving the storage space.
5. Using Column Types from other Database Engines: To make it
easier to use code written for SQL implementations from other vendors,
MySQL maps column types. These mappings make it easier to import
table definitions from other database engines into MySQL.
Sikkim Manipal University
Page No. 49
Unit 2
2.10 Answers
Self Assessment Questions
1. strict
2. BLOB
3. temporal
4. ALLOW_INVALID_DATES
5. 'YYYY-MM-DD HH:MM:SS'
Terminal Questions
1. FLOAT [(M,D)] [UNSIGNED] [ZEROFILL]: A small (single-precision)
floating-point number. Allowable values are -3.402823466E+38 to 1.175494351E-38, 0, and
1.175494351E-38 to 3.402823466E+38.
DOUBLE [(M,D)] [UNSIGNED] [ZEROFILL]: A normal - size (doubleprecision)
floating-point
number.
Allowable
values
are
1.7976931348623157E+308 to -2.2250738585072014E-308, 0, and
2.2250738585072014E-308 to 1.7976931348623157E+308.
M is the total number of digits and D is the number of digits following the
decimal point.
(Refer Section 2.2)
2. The CHAR and VARCHAR types are declared with a length that
indicates the maximum number of characters you want to store. For
example, CHAR(30) can hold up to 30 characters.
The length of a CHAR column is fixed to the length that you declare
when you create the table. The length can be any value from 0 to 255.
Values in VARCHAR columns are variable-length strings. The length
can be specified as a value from 0 to 65,535. The effective maximum
Sikkim Manipal University
Page No. 50
Unit 2
Size
(bytes)
Range
Format
Used For
DATE
1000-01-01 to
9999-12-31
YYYY-MM-DD
Date Values
TIME
-838:59:59 to
838:59:59
HH:MM:SS
Time values
or durations
YEAR
1901 to 2155
YYYY
Year Values
DATETIME
1000-01-01
00:00:00 to
9999-12-31
23:59:59
YYYY-MM-DD
HH:MM:SS
Combined
date and time
values
TIMESTAMP
1970-01-01
00:00:01 to
2038-01-09
03:14:07
YYYYMMDDHHMMSS
Combined
date and time
values,
timestamps
Storage Required
(in bytes)
CHAR(M)
VARCHAR(M)
TINYBLOB, TINYTEXT
BLOB, TEXT
MEDIUMBLOB, MEDIUMTEXT
LONGBLOB, LONGTEXT
ENUM('value1','value2',...)
SET('value1','value2',...)
Page No. 51
Unit 3
Unit 3
Structure
3.1
Introduction
Objectives
3.2
CREATE DATABASE
3.3
CREATE INDEX
3.4
CREATE TABLE
3.5
ALTER DATABASE
3.6
ALTER TABLE
3.7
DROP DATABASE
3.8
DROP INDEX
3.9
DROP TABLE
3.10 DESCRIBE
3.11 Summary
3.12 Terminal Questions
3.13 Answers
3.1 Introduction
The Data Definition Language (DDL) in any database server is used to
define the structures used to store the data. The Data Definition language
presented in this unit is used to create indexes and tables within the
specified database. The methods used to alter or modify the existing data
structures like tables or indexes are discussed with appropriate examples.
To view the structure or metadata information regarding the created
structures in the database, we use the Describe command whose syntax is
presented
Objectives
After studying this units, you should be able to:
explain various Data Definition Language (DDL) Statements
apply various CREATE syntaxes to create the appropriate structures
apply various ALTER syntaxes to modify the appropriate structures
explain and apply the DROP syntaxes
explain the usage of various Describe syntaxes
Page No. 52
Unit 3
If you manually create a directory under the data directory (for example, with
mkdir), the server considers it as a database directory and is shown in the
output of SHOW_DATABASES.
Self Assessment Questions
1. ______________ is a synonym for CREATE DATABASE.
Page No. 53
Unit 3
times faster than reading sequentially. If you need to access most of the
rows, it is faster to read sequentially, because this minimizes disk seeks.
Most MySQL indexes (PRIMARY KEY, UNIQUE, INDEX, and FULLTEXT)
are stored in B-trees. Exceptions are that indexes on spatial data types use
R-trees, and that MEMORY tables also support hash indexes.
MySQL uses indexes for these operations:
To find the rows matching a WHERE clause quickly.
To eliminate rows from consideration. If there is a choice between
multiple indexes, MySQL normally uses the index that finds the smallest
number of rows.
To retrieve rows from other tables when performing joins. MySQL can
use indexes on columns more efficiently if they are declared as the
same type and size. In this context, VARCHAR and CHAR are
considered the same if they are declared as the same size. For
example, VARCHAR(10) and CHAR(10) are the same size, but
VARCHAR(10) and CHAR(15) are not.
Comparison of dissimilar columns may prevent use of indexes if values
cannot be compared directly without conversion. Suppose that a
numeric column is compared to a string column. For a given value such
as 1 in the numeric column, it might compare equal to any number of
values in the string column such as '1', ' 1', '00001', or '01.e1'. This rules
out use of any indexes for the string column.
To find the MIN() or MAX() value for a specific indexed column key_col.
This is optimized by a preprocessor that checks whether you are using
WHERE key_part_N = constant on all key parts that occur before
key_col in the index. In this case, MySQL does a single key lookup for
each MIN() or MAX() expression and replaces it with a constant. If all
expressions are replaced with constants, the query returns at once. For
example:
Page No. 54
Unit 3
If a multiple-column index exists on col1 and col2, the appropriate rows can
be fetched directly. If separate single-column indexes exist on col1 and col2,
the optimizer will attempt to use the Index Merge optimization, or attempt
to find the most restrictive index by deciding which index finds fewer rows
and using that index to fetch the rows.
Index Merge optimization: The Index Merge method is used to retrieve
rows with several range scans and to merge their results into one. The
merge can produce unions, intersections, or unions-of-intersections of its
underlying scans. This access method merges index scans from a single
table; it does not merge scans across multiple tables.
CREATE INDEX is mapped to an ALTER TABLE statement to create
indexes. CREATE INDEX cannot be used to create a PRIMARY KEY; use
ALTER TABLE instead. Normally, you create all indexes on a table at the
time the table itself is created with CREATE TABLE. CREATE INDEX
enables you to add indexes to existing tables. A column list of the form
(col1,col2,...) creates a multiple-column index. Index values are formed by
concatenating the values of the given columns.
Indexes can be created that use only the leading part of column values,
using col_name(length) syntax to specify an index prefix length:
Prefixes can be specified for CHAR, VARCHAR, BINARY, and
VARBINARY columns.
Sikkim Manipal University
Page No. 55
Unit 3
BLOB and TEXT columns also can be indexed, but a prefix length must
be given.
Prefix lengths are given in characters for non-binary string types and in
bytes for binary string types. That is, index entries consist of the first
length characters of each column value for CHAR, VARCHAR, and
TEXT columns, and the first length bytes of each column value for
BINARY, VARBINARY, and BLOB columns.
For spatial columns, prefix values cannot be given.
The statement shown here creates an index using the first 10 characters of
the name column:
CREATE INDEX part_of_name ON customer (name(10));
If names in the column usually differ in the first 10 characters, this index
should not be much slower than an index created from the entire name
column. Also, using column prefixes for indexes can make the index file
much smaller, which could save a lot of disk space and might also speed up
INSERT operations.
Prefix lengths are storage engine-dependent (for example, a prefix can be
up to 1000 bytes long for MyISAM tables, 767 bytes for InnoDB tables).
Note that prefix limits are measured in bytes, whereas the prefix length in
CREATE INDEX statements is interpreted as number of characters for nonbinary data types (CHAR, VARCHAR, TEXT). Take this into account when
specifying a prefix length for a column that uses a multi-byte character set.
FULLTEXT indexes are supported only for MyISAM tables and can include
only CHAR, VARCHAR, and TEXT columns. Indexing always happens over
the entire column; column prefix indexing is not supported and any prefix
length is ignored if specified.
In MySQL 5.1:
You can add an index on a column that can have NULL values only if
you are using the MyISAM, InnoDB, or MEMORY storage engine.
You can add an index on a BLOB or TEXT column only if you are using
the MyISAM, or InnoDB storage engine.
An index_col_name specification can end with ASC or DESC. These
keywords are allowed for future extensions for specifying ascending or
Sikkim Manipal University
Page No. 56
Unit 3
descending index value storage. Currently, they are parsed but ignored;
index values are always stored in ascending order.
Self Assessment Questions
2. Database characteristics are stored in the ___ file in the database
directory.
3. BLOB and TEXT columns also can be indexed, but a ______ length
must be given.
Page No. 57
Unit 3
Purpose
Table format (definition)
file
Data file
Index file
OR
CREATE [TEMPORARY] TABLE [IF NOT EXISTS] tbl_name
[(create_definition,...)]
[table_option] ...
[partition_options]
select_statement
OR
CREATE [TEMPORARY] TABLE [IF NOT EXISTS] tbl_name
{ LIKE old_tbl_name | (LIKE old_tbl_name) }
Page No. 58
Unit 3
create_definition:
col_name column_definition
| [CONSTRAINT [symbol]] PRIMARY KEY [index_type] (index_col_name,...)
[index_option] ...
| {INDEX|KEY} [index_name] [index_type] (index_col_name,...)
[index_option] ...
| [CONSTRAINT [symbol]] UNIQUE [INDEX|KEY]
[index_name] [index_type] (index_col_name,...)
[index_option] ...
| {FULLTEXT|SPATIAL} [INDEX|KEY] [index_name] (index_col_name,...)
[index_option] ...
| [CONSTRAINT [symbol]] FOREIGN KEY
[index_name] (index_col_name,...) reference_definition
| CHECK (expr)
column_definition:
data_type [NOT NULL | NULL] [DEFAULT default_value]
[AUTO_INCREMENT] [UNIQUE [KEY] | [PRIMARY] KEY]
[COMMENT 'string'] [reference_definition]
[COLUMN_FORMAT {FIXED|DYNAMIC|DEFAULT}]
[STORAGE {DISK|MEMORY|DEFAULT}]
index_col_name:
col_name [(length)] [ASC | DESC]
index_type:
USING {BTREE | HASH | RTREE}
index_option:
KEY_BLOCK_SIZE [=] value
| index_type
| WITH PARSER parser_name
reference_definition:
REFERENCES tbl_name (index_col_name,...)
[MATCH FULL | MATCH PARTIAL | MATCH SIMPLE]
[ON DELETE reference_option]
[ON UPDATE reference_option]
reference_option:
RESTRICT | CASCADE | SET NULL | NO ACTION
Page No. 59
Unit 3
select_statement:
[IGNORE | REPLACE] [AS] SELECT ... (Some legal select statement)
Page No. 60
Unit 3
To make MySQL compatible with some ODBC applications, you can find the
AUTO_INCREMENT value for the last inserted row with the following query:
SELECT * FROM tbl_name WHERE auto_col IS NULL
Page No. 61
Unit 3
Page No. 62
Unit 3
MySQL optimizer to prioritize which index to use and also more quickly
to detect duplicated UNIQUE keys.
A PRIMARY KEY can be a multiple-column index. However, you cannot
create a multiple-column index using the PRIMARY KEY key attribute in
a column specification. Doing so only marks that single column as
primary. You must use a separate PRIMARY KEY(index_col_name, ...)
clause.
If a PRIMARY KEY or UNIQUE index consists of only one column that
has an integer type, you can also refer to the column as _rowid in
SELECT statements.
In MySQL, the name of a PRIMARY KEY is PRIMARY. For other
indexes, if you do not assign a name, the index is assigned the same
name as the first indexed column, with an optional suffix (_2, _3, ...) to
make it unique. You can see index names for a table using SHOW
INDEX FROM tbl_name.
Some storage engines allow you to specify an index type when creating
an index. The syntax for the index_type specifier is USING type_name.
Example:
CREATE TABLE lookup
(id INT, INDEX USING BTREE (id))
ENGINE = MEMORY;
Page No. 63
Unit 3
Page No. 64
Unit 3
BLOB and TEXT columns count from one to four plus eight bytes each
toward the row-size limit because their contents are stored separately.
Declaring columns NULL can reduce the maximum number of columns
allowed. NULL columns require additional space in the row to record
whether or not their values are NULL.
For MyISAM tables, each NULL column takes one bit extra, rounded up
to the nearest byte.
The maximum row length in bytes can be calculated as follows:
Row length = 1
+ (sum of column lengths)
+ (number of NULL columns + delete_flag + 7)/8
+ (number of variable-length columns)
delete_flag is 1 for tables with static row format. Static tables use a bit in the
row record for a flag that indicates whether the row has been deleted.
delete_flag is 0 for dynamic tables because the flag is stored in the dynamic
row header.
These calculations do not apply for InnoDB tables, for which storage size is
no different for NULL columns than for NOT NULL columns.
Each table has an .frm file that contains the table definition. The .frm file
size limit is fixed at 64KB. If a table definition reaches this size, no more
columns can be added. The expression that checks information to be
stored in the .frm file against the limit looks like this:
if (info_length+(ulong) create_fields.elements*FCOMP+288+
n_length+int_length+com_length > 65535L || int_count > 255)
FCOMP is 17.
n_length is the total length of all column names, including one byte
per name as a separator.
int_length is related to the list of values for SET and ENUM columns.
Page No. 65
Unit 3
Thus, using long column names can reduce the maximum number of
columns, as can the inclusion of ENUM or SET columns, or use of
column or table comments.
The TABLESPACE and STORAGE table options were both introduced in
MySQL 5.1.6. In MySQL 5.1, they are employed only with NDBCLUSTER
tables. The tablespace named tablespace_name must already have been
created using CREATE TABLESPACE. STORAGE determines the type of
storage used (disk or memory), and can be one of DISK, MEMORY, or
DEFAULT.
TABLESPACE ... STORAGE DISK assigns a table to a MySQL Cluster Disk
Data tablespace.
Important Note: A STORAGE clause cannot be used in a CREATE TABLE
statement without a TABLESPACE clause.
The ENGINE table option specifies the storage engine for the table.
The ENGINE table option takes the storage engine names shown in the
table 3.1.
Table 3.1: Storage Engines and Descriptions
Storage Engine
ARCHIVE
CSV
EXAMPLE
FEDERATED
HEAP
ISAM (OBSOLETE)
InnoDB
MEMORY
Description
The archiving storage engine
Tables that store rows in comma-separated values
format.
An example engine.
Storage engine that accesses remote tables.
This is a synonym for MEMORY.
Not available in MySQL 5.1. If you are upgrading to
MySQL 5.1 from a previous version, you should convert
any existing ISAM tables to MyISAM before performing
the upgrade.
Transaction-safe tables with row locking and foreign
keys.
The data for this storage engine is stored only in
memory.
Page No. 66
Unit 3
MERGE
MyISAM
NDBCLUSTER
If a storage engine is specified that is not available, MySQL uses the default
engine instead. Normally, this is MyISAM. For example, if a table definition
includes the ENGINE=INNODB option but the MySQL server does not
support INNODB tables, the table is created as a MyISAM table. This makes
it possible to have a replication setup where you have transactional tables
on the master but tables created on the slave are non-transactional (to get
more speed). In MySQL 5.1, a warning occurs if the storage engine
specification is not honored.
Engine substitution can be controlled by the setting of the
NO_ENGINE_SUBSTITUTION SQL mode.
The other table options are used to optimize the behavior of the table. In
most cases, you do not have to specify any of them. These options apply to
all storage engines unless otherwise indicated. Options that do not apply to
a given storage engine may be accepted and remembered as part of the
table definition. Such options then apply if you later use ALTER TABLE to
convert the table to use a different storage engine.
AUTO_INCREMENT: The initial AUTO_INCREMENT value for the
table. In MySQL 5.1, this works for MyISAM, MEMORY, and InnoDB
tables. It also works for ARCHIVE tables as of MySQL 5.1.6. To set the
first auto-increment value for engines that do not support the
AUTO_INCREMENT table option, insert a dummy row with a value one
less than the desired value after creating the table, and then delete the
dummy row.
For engines that support the AUTO_INCREMENT table option in
CREATE TABLE statements, you can also use ALTER TABLE tbl_name
AUTO_INCREMENT = N to reset the AUTO_INCREMENT value. The
Sikkim Manipal University
Page No. 67
Unit 3
value cannot be set lower than the maximum value currently in the
column.
AVG_ROW_LENGTH: An approximation of the average row length for
your table. You need to set this only for large tables with variable-size
rows.
When you create a MyISAM table, MySQL uses the product of the
MAX_ROWS and AVG_ROW_LENGTH options to decide how big the
resulting table is. If you don't specify either option, the maximum size for
MyISAM data and index files is 256TB by default. (If your operating
system does not support files that large, table sizes are constrained by
the file size limit.) If you want to keep down the pointer sizes to make the
index smaller and faster and you don't really need big files, you can
decrease
the
default
pointer
size
by
setting
the
myisam_data_pointer_size system variable. If you want all your tables to
be able to grow above the default limit and are willing to have your
tables slightly slower and larger than necessary, you can increase the
default pointer size by setting this variable. Setting the value to 7 allows
table sizes up to 65,536TB.
[DEFAULT] CHARACTER SET: Specify a default character set for the
table. CHARSET is a synonym for CHARACTER SET. If the character
set name is DEFAULT, the database character set is used.
CHECKSUM: Set this to 1 if you want MySQL to maintain a live
checksum for all rows (that is, a checksum that MySQL updates
automatically as the table changes). This makes the table a little slower
to update, but also makes it easier to find corrupted tables. The
CHECKSUM TABLE statement reports the checksum. (MyISAM only.)
[DEFAULT] COLLATE: Specify a default collation for the table.
COMMENT: A comment for the table, up to 60 characters long.
CONNECTION: The connection string for a FEDERATED table.
DATA DIRECTORY, INDEX DIRECTORY: By using DATA
DIRECTORY='directory' or INDEX DIRECTORY='directory' you can
specify where the MyISAM storage engine should put a table's data file
and index file. The directory must be the full pathname to the directory,
not a relative path.
Sikkim Manipal University
Page No. 68
Unit 3
Page No. 69
Unit 3
If you do not use PACK_KEYS, the default is to pack strings, but not
numbers. If you use PACK_KEYS=1, numbers are packed as well.
When packing binary number keys, MySQL uses prefix compression:
o Every key needs one extra byte to indicate how many bytes of the
previous key are the same for the next key.
o The pointer to the row is stored in high-byte-first order directly after
the key, to improve compression.
This means that if you have many equal keys on two consecutive rows, all
following same keys usually only take two bytes (including the pointer to
the row). Compare this to the ordinary case where the following keys takes
storage_size_for_key + pointer_size (where the pointer size is usually 4).
Conversely, you get a significant benefit from prefix compression only if you
have many numbers that are the same. If all keys are totally different, you
use one byte more per key, if the key is not a key that can have NULL
values. (In this case, the packed key length is stored in the same byte that is
used to mark if a key is NULL.)
PASSWORD: This option is unused. If you have a need to scramble
your .frm files and make them unusable to any other MySQL server,
please contact our sales department.
RAID_TYPE: RAID support has been removed as of MySQL 5.0.
ROW_FORMAT: Defines how the rows should be stored. For MyISAM
tables, the option value can be FIXED or DYNAMIC for static or variablelength row format. myisampack sets the type to COMPRESSED.
Note: When executing a CREATE TABLE statement, if you specify a row
format which is not supported by the storage engine that is used for the
table, the table is created using that storage engine's default row format.
The information reported in this column in response to SHOW TABLE
STATUS is the actual row format used. This may differ from the value in the
Create_options column because the original CREATE TABLE definition is
retained during creation.
UNION: UNION is used when you want to access a collection of
identical MyISAM tables as one. This works only with MERGE tables.
Page No. 70
Unit 3
You must have SELECT, UPDATE, and DELETE privileges for the
tables you map to a MERGE table.
partition_options can be used to control partitioning of the table created
with CREATE TABLE.
If used, a partition_options clause begins with PARTITION BY. This
clause contains the function that is used to determine the partition; the
function returns an integer value ranging from 1 to num, where num is
the number of partitions. (The maximum number of user-defined
partitions which a table may contain is 1024; the number of subpartitions
discussed later in this section is included in this maximum.) The
choices that are available for this function in MySQL 5.1 are shown in
the following list:
o HASH(expr): Hashes one or more columns to create a key for
placing and locating rows. expr is an expression using one or more
table columns. This can be any legal MySQL expression (including
MySQL functions) that yields a single integer value.
o KEY(column_list): This is similar to HASH, except that MySQL
supplies the hashing function so as to guarantee an even data
distribution. The column_list argument is simply a list of table
columns.
o For tables that are partitioned by key, you can employ linear
partitioning by using the LINEAR keyword. This has the same effect
as with tables that are partitioned by HASH. That is, the partition
number is found using the & operator rather than the modulus. You
may not use either VALUES LESS THAN or VALUES IN clauses
with PARTITION BY KEY.
o RANGE: In this case, expr shows a range of values using a set
of VALUES LESS THAN operators. When using range
partitioning, you must define at least one partition using VALUES
LESS THAN. You cannot use VALUES IN with range partitioning.
Page No. 71
Unit 3
Page No. 72
Unit 3
Page No. 73
Unit 3
MySQL creates new columns for all elements in the SELECT. For
example:
mysql>
CREATE
TABLE
test
(a
INT
NOT
NULL
AUTO_INCREMENT,
-> PRIMARY KEY (a), KEY(b))
-> ENGINE=MyISAM SELECT b,c FROM test2;
Page No. 74
Unit 3
For each row in table foo, a row is inserted in bar with the values
from foo and default values for the new columns.
In a table resulting from CREATE TABLE ... SELECT, columns
named only in the CREATE TABLE part come first. Columns named
in both parts or only in the SELECT part come after that. The data
type of SELECT columns can be overridden by also specifying the
column in the CREATE TABLE part.
If any errors occur while copying the data to the table, it is
automatically dropped and not created.
CREATE TABLE ... SELECT does not automatically create any
indexes for you. This is done intentionally to make the statement as
flexible as possible. If you want to have indexes in the created table,
you should specify these before the SELECT statement:
mysql> CREATE TABLE bar (UNIQUE (n)) SELECT n FROM foo;
Page No. 75
Unit 3
Page No. 76
Unit 3
The CHARACTER SET clause changes the default database character set.
The COLLATE clause changes the default database collation.
You can see what character sets and collations are available using,
respectively, the SHOW CHARACTER SET and SHOW COLLATION
statements.
The database name can be omitted from the first syntax, in which case the
statement applies to the default database.
The syntax that includes the UPGRADE DATA DIRECTORY NAME clause
was added in MySQL 5.1.23. It updates the name of the directory
associated with the database to use the encoding implemented in MySQL
5.1 for mapping database names to database directory names. This clause
is for use under these conditions:
It is intended when upgrading MySQL to 5.1 or later from older
versions.
It is intended to update a database directory name to the current
encoding format if the name contains special characters that need
encoding.
The statement is used by mysqlcheck (as invoked by
mysql_upgrade).
For example,if a database in MySQL 5.0 has a name of a-b-c, the name
contains instance of the - character. In 5.0, the database directory is also
Sikkim Manipal University
Page No. 77
Unit 3
named a-b-c, which is not necessarily safe for all filesystems. In MySQL 5.1
and up, the same database name is encoded as a@002db@002dc to
produce a filesystem-neutral directory name.
The action component here can be any of the keywords ADD, DROP,
ALTER, or CHANGE, and is followed by a field definition similar to that used
by the CREATE TABLE command. This definition consists of the name of
the field to be modified and (depending on the operation) a field definition
consisting of a new field name, type, and constraints.
Given below is an example which first creates a table and then adds a new
column to it.
Page No. 78
Unit 3
The following example adds a primary key column to the existing members
table:
Page No. 79
Unit 3
The following example changes the name and type of an existing field:
You can use the ALTER TABLE command to rename a table as shown
below:
Page No. 80
Unit 3
You can also use the equivalent RENAME TABLE command as shown
below:
You can set (or remove) a default value for a field with the SET DEFAULT
and DROP DEFAULT clauses, and you can control the position of fields with
the FIRST and AFTER clauses to the ALTER TABLE command.
Page No. 81
Unit 3
Page No. 82
Unit 3
Now, if you were to set the name field to be UNIQUE, MySQL would return
an error about duplicate values in that field.
Page No. 83
Unit 3
Page No. 84
Unit 3
.MRG
.TRN
.ndb
Page No. 85
Unit 3
so be careful with this statement! If any of the tables named in the argument
list do not exist, MySQL returns an error indicating by name which nonexisting tables it was unable to drop, but it also drops all of the tables in the
list that do exist.
DROP [TEMPORARY] TABLE [IF EXISTS]
tbl_name [, tbl_name] ...
[RESTRICT | CASCADE]
Important Note: When a table is dropped, user privileges on the table are
not automatically dropped.
Note that for a partitioned table, DROP TABLE permanently removes the
table definition, all of its partitions, and all of the data which was stored in
those partitions. It also removes the partitioning definition (.par) file
associated with the dropped table.
Use IF EXISTS to prevent an error from occurring for tables that do not
exist. A NOTE is generated for each non-existent table when using IF
EXISTS.
RESTRICT and CASCADE are allowed to make porting easier. In MySQL
5.1, they do nothing.
Note: DROP TABLE automatically commits the current active transaction,
unless you use the TEMPORARY keyword.
The TEMPORARY keyword has the following effects:
The statement drops only TEMPORARY tables.
The statement does not end an ongoing transaction.
No access rights are checked. (A TEMPORARY table is visible only to
the client that created it, so no check is necessary.)
Using TEMPORARY is a good way to ensure that you do not accidentally
drop a non-TEMPORARY table.
Page No. 86
Unit 3
3.11 Summary
This unit covers the following topics with respect to data definition language
statements in MySQL:
1. CREATE DATABASE: This topic covers the starting point of creating
the databases before any data objects are built into the databases. The
user must first create a database and build objects like tables, views,
stored procedures and so on inside the database. It gives the syntax
associated with creation of these databases within the MySQL server.
2. CREATE INDEX: The importance and syntaxes used in the creation of
indexes is mentioned in this topic.
Sikkim Manipal University
Page No. 87
Unit 3
3. CREATE TABLE: Tables are the basic data structures used to store
data. The syntaxes and variations of creating the tables are discussed
here.
4. ALTER DATABASE: Database modifications and the respective
syntaxes are described along with suitable examples are discussed.
5. ALTER TABLE: Modifications to the existing tables and the
corresponding syntaxes are described with suitable examples are
discussed here.
6. DROP DATABASE: Databases can be dropped as and when required.
The details of this is discussed.
7. DROP INDEX: Indexes can be dropped depending on the requirement
and size of the database objects.
8. DROP TABLE: Tables Indexes can be dropped depending on the
requirement of the user or application.
9. DESCRIBE: Anyone who needs to know the structure and data types of
the database tables can use this command for this purpose.
3.13 Answers
Self Assessment Questions
1. CREATE SCHEMA
2. db.opt
3. prefix
4. TEMPORARY
5. DEFAULT
6. UNIQUE
7. .frm
Page No. 88
Unit 3
Terminal Questions
1.
To find the rows matching a WHERE clause quickly.
To eliminate rows from consideration.
To retrieve rows from other tables when performing joins.
To find the MIN() or MAX() value for a specific indexed column
key_col.
(Refer Section 3.3)
2.
ALTER DATABASE enables you to change the overall characteristics of a
database. These characteristics are stored in the db.opt file in the database
directory. To use ALTER DATABASE, you need the ALTER privilege on the
database. ALTER SCHEMA is a synonym for ALTER DATABASE.
ALTER {DATABASE | SCHEMA} [db_name]
alter_specification ...
ALTER {DATABASE | SCHEMA} db_name
UPGRADE DATA DIRECTORY NAME
alter_specification:
[DEFAULT] CHARACTER SET [=] charset_name
| [DEFAULT] COLLATE [=] collation_name
The CHARACTER SET clause changes the default database character set.
The COLLATE clause changes the default database collation.
You can see what character sets and collations are available using,
respectively, the SHOW CHARACTER SET and SHOW COLLATION
statements.
The database name can be omitted from the first syntax, in which case the
statement applies to the default database.
(Refer Section 3.5)
3.
a. CREATE TABLE members(mid INT(3), mname CHAR(8), mpass
VARCHAR(25));
ALTER TABLE members ADD email VARCHAR(255) NOT NULL;
b. ALTER TABLE members ADD PRIMARY KEY(email);
c. ALTER TABLE members CHANGE mid id INT(8) AUTO_INCREMENT
UNIQUE;
(Refer Section 3.6)
Sikkim Manipal University
Page No. 89
Unit 4
Unit 4
Structure
4.1 Introduction
Objectives
4.2 Loading Data into a Table
INSERT
INSERT ... SELECT
INSERT DELAYED
LOAD DATA INFILE
4.3 Retrieving Information from a Table
Retrieving Specific Rows and Columns
Sorting Query Results
Grouping Query Results
4.4 Summary
4.5 Terminal Questions
4.6 Answers
4.1 Introduction
The Data Manipulation (DML) statements are used to manipulate the data
from the underlying data structures like tables, views, functions and so on.
They follow the standard SQL norms. There are many Data Manipulation
statements like Insert, Delete Load, and Update pertaining to MySQL.
These DML statements are used commonly for the operations mentioned
above. In this unit we are going to have the syntaxes and the corresponding
examples discussed in detail. This unit takes a walkthrough of all the DML
statements.
Objectives
After studying this unit, you should be able to:
explain the importance of DML (Data Manipulation Language)
Statements
explain various methods of loading data into tables in MySQL
describe various methods used to select the data
describe the various ways of grouping and sorting of the output data
Page No. 90
Unit 4
Now we will use the INSERT statement to put some data into the table.
Page No. 91
Unit 4
The table should now contain a single record for Rob Rabbit. To verify this
you can issue a simple select statement on the table as follows:
You can also use an abbreviated form of the INSERT statement, in which
the field list is left unspecified. The following example, an equivalent of the
previous INSERT statement is given below:
mysql> INSERT INTO addressbook VALUES(Rob, Rabbit, 674 1536,
382 8364, rob@some.domain);
Note: When using this shorter format, the order in which values are inserted
must correspond to the sequence of fields in the table, which can be easily
determined by issuing the DESCRIBE command on the respective table.
We can also INSERT the results of calculations and function calls as below:
Page No. 92
Unit 4
Page No. 93
Unit 4
Note:
1. When inserting string values (and some date values) into a table, you
must always surround them with quotation marks or MySQL will treat
them as field names.
2. Numeric values need not be quoted.
MySQL also supports the following two non-standard variants of the
INSERT statement:
1. Multiple records can be entered in a single INSERT statement through
the use of multiple VALUES() clauses within the same statement. For
example instead of doing this,
Page No. 94
Unit 4
You can also do away with the INSERTVALUES form altogether in favor
of syntax similar to that used by the UPDATE statement. This involves
using the SET clause to set values for each column individually.
So instead of doing this,
Page No. 95
Unit 4
Page No. 96
Unit 4
MySQL 4.0.3 and higher also support a new DEFAULT keyword, which can
be used to instruct MySQL to use the columns default value for that record.
Page No. 97
Unit 4
Example:
INSERT INTO tbl_temp2 (fld_id)
SELECT tbl_temp1.fld_order_id
FROM tbl_temp1 WHERE tbl_temp1.fld_order_id > 100;
Page No. 98
Unit 4
table to hold the rows from the SELECT and then inserts those rows into
the target table. However, it remains true that you cannot use INSERT
INTO t ... SELECT ... FROM t when t is a TEMPORARY table, because
TEMPORARY tables cannot be referred to twice in the same statement.
AUTO_INCREMENT columns work as usual.
To ensure that the binary log can be used to re-create the original
tables, MySQL does not allow concurrent inserts for INSERT ... SELECT
statements.
Currently, you cannot insert into a table and select from the same table
in a subquery.
To avoid ambiguous column reference problems when the SELECT and
the INSERT refer to the same table, provide a unique alias for each
table used in the SELECT part, and qualify column names in that part
with the appropriate alias.
In the values part of ON DUPLICATE KEY UPDATE, you can refer to
columns in other tables, as long as you do not use GROUP BY in the
SELECT part. One side effect is that you must qualify non-unique column
names in the values part.
4.2.3 INSERT DELAYED Syntax
The DELAYED option for the INSERT statement is a MySQL extension to
standard SQL that is very useful if you have clients that cannot or need not
wait for the INSERT to complete. This is a common situation when you use
MySQL for logging and you also periodically run SELECT and UPDATE
statements that take a long time to complete.
When a client uses INSERT DELAYED, it gets an okay from the server at
once, and the row is queued to be inserted when the table is not in use by
any other thread.
Another major benefit of using INSERT DELAYED is that inserts from many
clients are bundled together and written in one block. This is much faster
than performing many separate inserts.
Note that INSERT DELAYED is slower than a normal INSERT if the table is
not otherwise in use. There is also the additional overhead for the server to
handle a separate thread for each table for which there are delayed rows.
This means that you should use INSERT DELAYED only when you are
really sure that you need it.
Sikkim Manipal University
Page No. 99
Unit 4
The queued rows are held only in memory until they are inserted into the
table. This means that if you terminate mysqld forcibly (for example, with kill
-9) or if mysqld dies unexpectedly, any queued rows that have not been
written to disk are lost.
Syntax:
INSERT DELAYED ...
Unit 4
preferable to specify the character set of the file by using the CHARACTER
SET clause, which is available as of MySQL 5.1.17.
LOAD DATA INFILE interprets all fields in the file as having the same
character set, regardless of the data types of the columns into which field
values are loaded. For proper interpretation of file contents, you must
ensure that it was written with the correct character set. For example, if you
write a data file with mysqldump -T or by issuing a SELECT ... INTO
OUTFILE statement in mysql, be sure to use a default-character-set
option with mysqldump or mysql so that output is written in the character
set to be used when the file is loaded with LOAD DATA INFILE.
Note: It is currently not possible to load data files that use the ucs2
character set.
You can also load data files by using the mysqlimport utility; it operates by
sending a LOAD DATA INFILE statement to the server. The local option
causes mysqlimport to read data files from the client host. You can specify
the compress option to get better performance over slow networks if the
client and server support the compressed protocol.
If you use LOW_PRIORITY, execution of the LOAD DATA statement is
delayed until no other clients are reading from the table. This affects only
storage engines that use only table-level locking (MyISAM, MEMORY,
MERGE).
If you specify CONCURRENT with a MyISAM table that satisfies the
condition for concurrent inserts (that is, it contains no free blocks in the
middle), other threads can retrieve data from the table while LOAD DATA is
executing. Using this option affects the performance of LOAD DATA a bit,
even if no other thread is using the table at the same time.
CONCURRENT is not replicated when using statement-based replication;
however, it is replicated when using row-based replication.
The LOCAL keyword, if specified, is interpreted with respect to the client
end of the connection:
If LOCAL is specified, the file is read by the client program on the client
host and sent to the server. The file can be given as a full pathname to
specify its exact location. If given as a relative pathname, the name is
interpreted relative to the directory in which the client program was
started.
Sikkim Manipal University
Unit 4
If LOCAL is not specified, the file must be located on the server host and
is read directly by the server. The server uses the following rules to
locate the file:
o If the filename is an absolute pathname, the server uses it as given.
o If the filename is a relative pathname with one or more leading
components, the server searches for the file relative to the server's
data directory.
o If a filename with no leading components is given, the server looks
for the file in the database directory of the default database.
Note that, in the non-LOCAL case, these rules mean that a file named as
./myfile.txt is read from the server's data directory, whereas the file named
as myfile.txt is read from the database directory of the default database. For
example, if db1 is the default database, the following LOAD DATA
statement reads the file data.txt from the database directory for db1, even
though the statement explicitly loads the file into a table in the db2
database:
LOAD DATA INFILE 'data.txt' INTO TABLE db2.my_table;
Unit 4
If you want to ignore foreign key constraints during the load operation, you
can issue a SET FOREIGN_KEY_CHECKS=0 statement before executing
LOAD DATA.
In some extreme cases, you can create the indexes even faster by turning
them off with ALTER TABLE ... DISABLE KEYS before loading the file into
the table and using ALTER TABLE ... ENABLE KEYS to re-create the
indexes after loading the file.
For both the LOAD DATA INFILE and SELECT ... INTO OUTFILE
statements, the syntax of the FIELDS and LINES clauses is the same. Both
clauses are optional, but FIELDS must precede LINES if both are specified.
If you specify a FIELDS clause, each of its subclauses (TERMINATED BY,
[OPTIONALLY] ENCLOSED BY, and ESCAPED BY) is also optional,
except that you must specify at least one of them.
If you specify no FIELDS clause, the defaults are the same as if you had
written this:
FIELDS TERMINATED BY '\t' ENCLOSED BY '' ESCAPED BY '\\'
If you specify no LINES clause, the defaults are the same as if you had
written this:
LINES TERMINATED BY '\n' STARTING BY ''
The defaults cause LOAD DATA INFILE to act as follows when reading
input:
Look for line boundaries at new lines.
Do not skip over any line prefix.
Break lines into fields at tabs.
Do not expect fields to be enclosed within any quoting characters.
Interpret occurrences of tab, newline, or \ preceded by \ as literal
characters that are part of field values.
Conversely, the defaults cause SELECT ... INTO OUTFILE to act as
follows when writing output:
Write tabs between fields.
Do not enclose fields within any quoting characters.
Use \ to escape instances of tab, newline, or \ that occur within field
values.
Write newlines at the ends of lines.
Sikkim Manipal University
Unit 4
Unit 4
Unit 4
In general, clauses used must be given in exactly the order shown in the
syntax description. For example, a HAVING clause must come after any
GROUP BY clause and before any ORDER BY clause. The exception is
that the INTO clause can appear either as shown in the syntax description
or immediately following the select_expr list.
A select_expr can be given an alias using AS alias_name. The alias is used
as the expression's column name and can be used in GROUP BY, ORDER
BY, or HAVING clauses.
The AS keyword is optional when aliasing a select_expr. It is good practice
to be in the habit of using AS explicitly when specifying column aliases.
The FROM table_references clause indicates the table or tables from which
to retrieve rows. If you name more than one table, you are performing a join.
For each table specified, you can optionally specify an alias.
tbl_name [[AS] alias] [index_hint]
The use of index hints provides the optimizer with information about how to
choose indexes during query processing.
You can use SET max_seeks_for_key=value as an alternative way to force
MySQL to prefer key scans instead of table scans.
You can refer to a table within the default database as tbl_name, or as
db_name.tbl_name to specify a database explicitly. You can refer to a
column as col_name, tbl_name.col_name, or db_name.tbl_name.col_name.
Sikkim Manipal University
Unit 4
To sort in reverse order, add the DESC (descending) keyword to the name
of the column in the ORDER BY clause that you are sorting by. The default
is ascending order; this can be specified explicitly using the ASC keyword.
If ORDER BY occurs within a subquery and also is applied in the outer
query, the outermost ORDER BY takes precedence. For example, results
for the following statement are sorted in descending order, not ascending
order:
(SELECT ... ORDER BY a) ORDER BY a DESC;
If you use GROUP BY, output rows are sorted according to the GROUP BY
columns as if you had an ORDER BY for the same columns. To avoid the
overhead of sorting that GROUP BY produces, add ORDER BY NULL:
SELECT a, COUNT(b) FROM test_table GROUP BY a ORDER BY NULL;
MySQL extends the GROUP BY clause so that you can also specify ASC
and DESC after columns named in the clause:
SELECT a, COUNT(b) FROM test_table GROUP BY a DESC;
Unit 4
Unit 4
The following query retrieves a list of all accounts created in the year 2002.
The LIMIT keyword can be used to restrict the total number of records
returned by a SELECT query.
Example: The following query restricts the result set to 5 records.
Unit 4
Example: The following example sorts the query results on accounts table
by creation date.
The ASC and DESC keywords can be added to each field name in the
ORDER BY clause to customize the sorting methods further.
4.3.3 Grouping Query Results
MySQL makes it possible to break the records in a result set into distinct
groups on the basis of a specific attribute with the GROUP BY clause.
Because each group created in this manner is represented as a single row
(even though it contains multiple records), this capability is primarily used in
operations involving MySQLs numerous aggregate functions.
Example: The records in the accounts table are to be grouped on the basis
of acccreationplace. There are 5 distinct locations spread over the 11
records in the table. MySQL creates 5 distinct groups of records and
represents each group as a single row in the result set.
Unit 4
Unit 4
4.4 Summary
This unit covers the following topics with respect to data manipulation
language statements in MySQL:
1. Loading Data into a table: Data loading into tables either from the user
through SQL statements or through external resources or from some
other existent tables is discussed here.
2. Retrieving information from a Table: Information retrieval is a major
task for the database users. This can be done in many ways through the
usage of appropriate SELECT statements in conjunction with clauses
like GROUP BY or ORDER BY.
4.6 Answers
Self Assessment Questions
1. AUTOINCREMENT
2. HAVING
3. outermost
Terminal Questions
1.
Syntax:
INSERT INTO table-name (field-name, field-name, )
VALUES (field-value, field-value, )
Unit 4
Example:
Create a table called student as follows:
CREATE TABLE student(studno INT(5) PRIMARY KEY, Studname
VARCHAR(30) NOT NULL);
The following Insert statement is used to insert values into student table:
INSERT INTO student(studno,studname) VALUES (10,abc);
INSERT INTO student(studno,studname) VALUES (20,def);
(Refer Section 4.2)
2.
a. SELECT 50<=(3 + 2);
b. SELECT t1.name, t2.salary FROM employee t1, info t2 WHERE
t1.name = t2.name;
c. SELECT college, region AS r, seed AS s FROM tournament ORDER
BY r, s;
(Refer Section 4.3)
3. SELECT accreationplace AS LOC, COUNT(*) FROM accounts
GROUP BY accreationplace;
(Refer Section 4.3)
Unit 5
5.1
5.2
5.3
5.4
5.5
5.6
5.7
5.8
5.9
5.10
5.11
5.12
Unit 5
Introduction
Objectives
JOIN
UNION
DELETE
TRUNCATE
UPDATE
DO
HANDLER
REPLACE
Summary
Terminal Questions
Answers
5.1 Introduction
This unit is an extension to the basic Data Manipulation statements
introduced in the previous unit. It discusses about the JOIN and UNION
statements used to combine two or more database tables or views in many
ways. It also describes the ways in which data or objects can be deleted
using the DELETE command. It describes the differences between
TRUNCATE and DELETE commands. The other DML statements like
UPDATE, DO, REPLACE and HANDLER are also discussed in detail.
Objectives
After studying this unit, you should be able to:
explain the various methods of joining tables with JOIN statement
explain the approach of using UNION operator in combining two or
more tables
distinguish and Differentiate between DELETE and TRUNCATE
Statements
describe the usage of UPDATE statement in modifying the existing
data
describe the DO syntax
describe the usage and applications of HANDLER and REPLACE
statements
Sikkim Manipal University
Unit 5
5.2 JOIN
Efficiency constraints usually dictate that data be split across multiple tables
and that relationships be created between different tables to make efficient
retrieval of data.
By supporting the creation of links between related pieces of information, a
Relational Database Management System (RDBMS) not only makes it
possible to store information more efficiently (by removing redundancies), it
also brings to the forefront undiscovered relationships between disparate
segments of data and permits efficient exploitation of those relationships.
This section demonstrates how SQL can be used to query multiple tables at
once and to combine the data retrieved from them in different ways.
These multi-table queries are referred to as JOINS because they join
together two or more tables.
MySQL has supported joins well right from its inception and, today, boasts
support for standard SQL2-compliant Join syntax, which makes it possible
to combine table records in a variety of sophisticated ways.
Consider the following sample tables that would be used to demonstrate the
concept of Joins.
The first table named Categories which contains a list of news categories is
structured as follows:
Unit 5
The second table named Headlines which contains a list of news headlines
is structured as follows:
A link exists between the previous two tables: the cid field, which be used to
connect each news item with its category.
Unit 5
The following query retrieves all the records from the table topics:
Unit 5
Types of Joins:
INNER
OUTER (LEFT, RIGHT, FULL)
CROSS
- INNER JOIN: They are also known as Equi Joins. They are so called
because the where statement generally compares two columns from two
tables with the equivalence operator =. Many systems use this type as the
default join. This type can be used in situations where selecting only those
rows that have values in common in the columns specified in the ON clause,
is required. In short, the Inner Join returns all rows from both tables where
there is a match.
Products
ProdID
1
2
3
4
Sales
ProdName
Apples
Oranges
Pineapples
Bananas
ID
1
2
3
ProdID
3
2
1
Quantity
2300
1500
3400
As both the tables have a ProdID field in common, it is easy to join them.
mysql> SELECT ProdName, Quantity FROM Products, Sales
WHERE Products.ProdID = Sales.ProdID AND
Sales.Quantity > 2000;
ProdName
Pineapples
Apples
Quantity
2300
3400
OUTER JOIN: This type of join can be used in situations where it is desired,
to select all rows from the table on the left (or right, or both) regardless of
whether the other table has values in common and (usually) enter NULL
where data is missing.
Unit 5
groups
name
Sue
Harry
Louis
Sam
James
Mark
Rita
gid
501
502
503
504
name
Authors
Actors
Musicians
chefs
users_groups
uid
11
107
100
110
112
100
102
104
100
gid
502
502
503
501
501
501
501
502
502
The following table shows the output of the query list which users belong to
which groups:
mysql> SELECT users.name, groups.name
FROM users, groups, users_groups
WHERE users.uid = users_groups.uid
AND groups.gid = users_groups.gid;
Sikkim Manipal University
name
Sam
Sue
James
Rita
Sue
Louis
Sue
Unit 5
name
Actors
Musicians
Authors
Authors
Authors
Actors
Actors
The above list is a symmetrical list, it doesnt tell you anything about the
users who doesnt belong to any group or the groups with no members.
Example: (Left Outer Join)
mysql> SELECT * FROM users LEFT JOIN users_groups ON
users.uid = users_groups.uid;
Select all rows from the left side of the join (table users) and, for each row
selected, either display the matching value (the value satisfying the
constraints in the ON or USING clause) from the right side (table
users_groups) or display a row of NULLS.
uid
100
100
100
103
104
107
110
111
112
name
Sue
Sue
Sue
Harry
Louis
Sam
James
Mark
rita
gid
503
501
502
NULL
502
502
501
NULL
501
uid
100
100
100
NULL
104
107
110
NULL
112
Unit 5
USING (gid);
gid
501
501
501
501
502
502
502
502
503
NULL
uid
110
112
100
102
11
107
104
100
100
NULL
gid
501
501
501
501
502
502
502
502
503
504
name
authors
authors
authors
authors
actors
actors
actors
actors
musicians
Chefs
CROSS JOIN: This type of join returns a Cartesian product. i.e. it combines
every row from the left table with every row in the right table. Sometimes this
join produces a mess, but under the right circumstances, it can be very
useful. This type of join can be used in situations where it is desired, to
select all possible combinations of rows and columns from both tables.
Attribute
Color
Attribute
Eyes
Hair
Color
Brown
Black
Gray
Attribute
Eyes
Eyes
Eyes
Hair
Hair
Hair
Page No. 121
Unit 5
5.3 UNION
SELECT ...
UNION [ALL | DISTINCT] SELECT ...
[UNION [ALL | DISTINCT] SELECT ...]
UNION is used to combine the result from multiple SELECT statements into
a single result set.
The column names from the first SELECT statement are used as the
column names for the results returned. Selected columns listed in
corresponding positions of each SELECT statement should have the same
data type. (For example, the first column selected by the first statement
should have the same type as the first column selected by the other
statements.)
If the data types of corresponding SELECT columns do not match, the types
and lengths of the columns in the UNION result take into account the values
retrieved by all of the SELECT statements.
The default behavior for UNION is that duplicate rows are removed from the
result. The optional DISTINCT keyword has no effect other than the default
because it also specifies duplicate-row removal. With the optional ALL
keyword, duplicate-row removal does not occur and the result includes all
matching rows from all the SELECT statements.
You can mix UNION ALL and UNION DISTINCT in the same query.
Unit 5
Example 1
Example 2
Example 3
Unit 5
5.4 DELETE
DELETE [LOW_PRIORITY] [QUICK] [IGNORE] FROM
tbl_name
[WHERE where_condition]
[ORDER BY ...]
[LIMIT row_count]
The DELETE statement deletes rows from tbl_name and returns a count of
the number of deleted rows. This count can be obtained by calling the
ROW_COUNT() function.
The WHERE clause, if given, specifies the conditions that identify which
rows to delete. With no WHERE clause, all rows are deleted.
If the ORDER BY clause is specified, the rows are deleted in the order that
is specified. The LIMIT clause places a limit on the number of rows that can
be deleted. As stated, a DELETE statement with no WHERE clause deletes
all rows.
A faster way to do this, when you do not need to know the number of
deleted rows, is to use TRUNCATE TABLE. However, within a transaction
or if you have a lock on the table, TRUNCATE TABLE cannot be used
whereas DELETE can.
Deleting Tables:
You can delete a table with the DROP TABLE command.
Unit 5
Example:
5.5 TRUNCATE
TRUNCATE [TABLE] tbl_name
Unit 5
Unit 5
5.6 UPDATE
UPDATE [LOW_PRIORITY] [IGNORE] tbl_name
SET col_name1={expr1|DEFAULT} [, col_name2={expr2|DEFAULT}] ...
[WHERE where_condition]
[ORDER BY ...]
[LIMIT row_count]
5.7 DO
Syntax:
DO expr [, expr] ...
Sikkim Manipal University
Unit 5
DO executes the expressions but does not return any results. In most
respects, DO is shorthand for SELECT expr, ..., but has the advantage that
it is slightly faster when you do not care about the result. DO is useful
primarily with functions that have side effects, such as RELEASE_LOCK().
5.8 HANDLER
HANDLER tbl_name OPEN [ [AS] alias]
HANDLER tbl_name READ index_name { = | >= | <= | < } (value1,value2,...)
[ WHERE where_condition ] [LIMIT ... ]
HANDLER tbl_name READ index_name { FIRST | NEXT | PREV | LAST }
[ WHERE where_condition ] [LIMIT ... ]
HANDLER tbl_name READ { FIRST | NEXT }
[ WHERE where_condition ] [LIMIT ... ]
HANDLER tbl_name CLOSE
Unit 5
three columns in the index, or for the columns in a leftmost prefix. For
example:
HANDLER ... READ my_idx = (col_a_val,col_b_val,col_c_val) ...
HANDLER ... READ my_idx = (col_a_val,col_b_val) ...
HANDLER ... READ my_idx = (col_a_val) ...
5.9 REPLACE
A subtle variation on the ON DUPLICATE KEY UPDATE is the REPLACE
command, which adopts the same syntax as the INSERT command. Unlike
INSERT, though, which produces an error if the record being inserted
contains a duplicate value on a filed marked as UNIQUE, REPLACE
replaces the entire record with new values.
The difference between ON DUPLICATE KEY UPDATE and REPLACE is
this: while the former only updates the named fields with new values, the
latter deletes the old record and replaces it completely with the new one.
Syntax 1:
REPLACE [LOW_PRIORITY | DELAYED]
[INTO] tbl_name [(col_name,...)]
{VALUES | VALUE} ({expr | DEFAULT},...),(...),...
OR
Syntax 2:
REPLACE [LOW_PRIORITY | DELAYED]
[INTO] tbl_name
SET col_name={expr | DEFAULT}, ...
Sikkim Manipal University
Unit 5
OR
Syntax 3:
REPLACE [LOW_PRIORITY | DELAYED]
[INTO] tbl_name [(col_name,...)]
SELECT ...
REPLACE works exactly like INSERT, except that if an old row in the table
has the same value as a new row for a PRIMARY KEY or a UNIQUE index,
the old row is deleted before the new row is inserted. REPLACE is a MySQL
extension to the SQL standard. It either inserts, or deletes and inserts.
Note that unless the table has a PRIMARY KEY or UNIQUE index, using a
REPLACE statement makes no sense. It becomes equivalent to INSERT,
because there is no index to be used to determine whether a new row
duplicates another.
Values for all columns are taken from the values specified in the REPLACE
statement. Any missing columns are set to their default values, just as
happens for INSERT. You cannot refer to values from the current row and
use them in the new row. If you use an assignment such as SET col_name
= col_name + 1, the reference to the column name on the right hand side is
treated as DEFAULT(col_name), so the assignment is equivalent to SET
col_name = DEFAULT(col_name) + 1.
To use REPLACE, you must have both the INSERT and DELETE privileges
for the table.
The REPLACE statement returns a count to indicate the number of rows
affected. This is the sum of the rows deleted and inserted. If the count is 1
for a single-row REPLACE, a row was inserted and no rows were deleted. If
the count is greater than 1, one or more old rows were deleted before the
new row was inserted. It is possible for a single row to replace more than
one old row if the table contains multiple unique indexes and the new row
duplicates values for different old rows in different unique indexes.
The affected-rows count makes it easy to determine whether REPLACE
only added a row or whether it also replaced any rows: Check whether the
count is 1 (added) or greater (replaced).
Sikkim Manipal University
Unit 5
5.10 Summary
This unit covers the advanced data manipulation language statements as
described below:
1. Join: This statement is used to join tables within a database using
different variations like Inner Join, Outer Join, Self Join and so on.
2. Union: This follows the mathematical principles of Joins used to join the
tables and different database objects.
3. Delete: This command is used to delete the data or data objects within a
database.
4. Truncate: This statement is used to empty a table completely. Logically,
this is equivalent to 5. Delete: This statement that deletes all rows, but
there are practical differences under some circumstances.
6. Update: This statement causes modifications to the existing data in
various database objects like tables, views, and so on.
7. DO: DO executes the expressions but does not return any results. In
most respects, DO is shorthand for SELECT expr, ..., but has the
advantage that it is slightly faster when you do not care about the result.
DO is useful primarily with functions that have side effects, such as
RELEASE_LOCK().
8. Handler: The HANDLER statement provides direct access to table
storage engine interfaces. It is available for MyISAM and InnoDB tables.
9. Replace: Adopts the same syntax as the INSERT command. Unlike
INSERT, though, which produces an error if the record being inserted
contains a duplicate value on a field marked as UNIQUE, REPLACE
replaces the entire record with new values.
Unit 5
5.12 Answers
Self Assessment Questions
1. multi-table
2. a (Select)
3. LIMIT
4. DELETE
5. SET
Terminal Questions
1.
a. Inner Join: In this join type the where statement generally compares
two columns from two tables with the equivalence operator =. Many
systems use this type as the default join. Also known as Equi-Join
b. Left Outer Join: In this type, all the records from the table on the left
side of the join and matching the WHERE clause in appear in the final
result set.
c. Right Outer Join: All the records matching the WHERE clause from the
table on the right appear.
2.
a. Union Syntax: (Refer Section 5.3)
SELECT ...
UNION [ALL | DISTINCT] SELECT ...
[UNION [ALL | DISTINCT] SELECT ...]
Unit 6
Unit 6
Subqueries
Structure
6.1 Introduction
Objectives
6.2 The Subquery as Scalar Operand
6.3 Comparisons Using Subqueries
6.4 Subqueries with ANY, IN, and SOME
6.5 Subqueries with ALL
6.6 Correlated Subqueries
6.7 EXISTS and NOT EXISTS
6.8 Row Subqueries
6.9 Subqueries in the FROM clause
6.10 Summary
6.11 Terminal Questions
6.12 Answers
6.1 Introduction
Normally, the query results are restricted through the addition of a WHERE
or HAVING clause, which contains one or more conditional expressions
used to filter out irrelevant records from the result set.
A situation may arise where in the conditional test used by one query
depends on the result generated by another query (or Subquery). In all such
cases, the results generated by one query depend on the data generated by
another, and the use of a constant value in the outer querys conditional test
becomes infeasible.
Prior to version 4.1 of MySQL, the only way to address this type of
requirement was to perform each query individually, and to use the data
from one in the clause of the other. MySQL 4.1 (and later) does away with
this by introducing one of the most frequently asked for features on the
MySQL mailing lists: subqueries.
A Subquery is exactly what it sounds like: a SELECT query that is
subordinate to another query. MySQL 4.1 (and latter) enables to you nest
queries within one another, and to use the result set generated by an inner
query within an outer one. As a result, instead of executing two (or more)
Sikkim Manipal University
Unit 6
separate queries, you execute a single query containing one (or more)
subqueries.
When it encounters such a nested query, MySQL begins with the innermost
query and moves outward and upward from it to the outermost (main) query.
The result set generated by each query on the journey is assigned to the
enclosing parent query, which is, in turn, executed, and its results are further
assigned to its parent.
A Subquery works just like a regular SELECT query, except that its result
set always consists of a single column containing one or more values. A
Subquery can be used anywhere an expression can be used, it must be
enclosed in parentheses, and, like a regular SELECT query, it must contain
a field list, a FROM clause with one or more table names, and optional
WHERE, HAVING, and GROUP BY clauses.
The main advantages of subqueries are:
They allow queries that are structured so that it is possible to isolate
each part of a statement.
They provide alternative ways to perform operations that would
otherwise require complex joins and unions.
They are, in many people's opinion, more readable than complex joins
or unions. Indeed, it was the innovation of subqueries that gave people
the original idea of calling the early SQL Structured Query Language.
Objectives
After studying this unit, you should be able to:
describe the usage of subquery as a scalar operand
explain how comparison of columns or tuples can be made using
subqueries
give examples of subqueries using the operators ALL, ANY, IN, and
SOME
solve the problems related to Correlated Subqueries
give examples of subqueries using the operators EXISTS and NOT
EXISTS
construct queries using Row Subqueries
demonstrate the usage of FROM clause in subqueries
Unit 6
Example: We need a list of all the branch offices belonging to Rabbit Foods
Inc. Now you could do this by running two SELECT queries, one after
another, to first get the customer ID of Rabbit Foods Inc., and then using
that ID (104) in another query to get the list of branch offices liked to that
customer,
Unit 6
OR with a Subquery:
Unit 6
Unit 6
And then filter out those with only two offices with a HAVING clause,
And then hand the client ID over to the clients table to get the client name.
Unit 6
The following subquery takes care of the previous three steps for you in one
go:
In this case, the inner query is executed first this query takes care of
grouping the branches by customer ID and counting the number of records
(branch offices) in each group. Those customers who have exactly two
branch offices can easily be filtered out with a HAVING clause, and the
corresponding customer IDs returned to the main query. This then maps the
IDs into the customers table and returns the corresponding customer name.
Example Select all those customers using the service with the maximum
service fee.
Unit 6
Then look up each service ID in the services table for the corresponding
name.
Unit 6
In the query above, MySQL will select only those records from the services
table that match the service ID collection returned by the subquery.
Unit 6
The following query shows the usage of the operator ANY with a subquery:
Unit 6
The word SOME is an alias for ANY. Thus, these two statements are the
same:
SELECT s1 FROM t1 WHERE s1 <> ANY (SELECT s1 FROM t2);
SELECT s1 FROM t1 WHERE s1 <> SOME(SELECT s1 FROM t2);
In general, tables containing NULL values and empty tables are edge
cases. When writing subquery code, always consider whether you have
taken those two possibilities into account.
NOT IN is an alias for <> ALL. Thus, these two statements are the same:
SELECT s1 FROM t1 WHERE s1 <> ALL (SELECT s1 FROM
t2);
SELECT s1 FROM t1 WHERE s1 NOT IN (SELECT s1 FROM
t2);
Unit 6
Unit 6
Example:
If you look at the data of the corresponding tables mentioned in the above
example query, you will see in fact, only one branch is using four or more
services (branch ID 1011).
In this case since the inner query contains a reference to a field in the outer
query, MySQL cannot run the inner query only once. Rather, it has to run it
over and over, once for every row in the outer table, substitute the value of
the named field from that row in the subquery, and then decide whether to
include that outer row in the final result set on the basis of whether the
corresponding subquery returns a result.
Self Assessment Questions
4. A reference by a subquery to a field in its enclosing query, is called an
___ reference.
Unit 6
In this case, because some branches are using four or more services, the
inner query will return a result set consisting of at least one row, the EXISTS
test will return true, and the outer query will be executed.
If the result set generated by the previous query is itself immaterial, we
could accomplish the same thing with the following query:
The EXISTS operator is most often used in the context of outer references.
Outer References: Most of the time, if a subquery is going to produce the
same result set every time it runs, it makes sense, performance wise, to run
Sikkim Manipal University
Unit 6
it only once and use the same result set to test every record generated in
the main query.
However, situations sometimes arise in which a subquery uses a field from
the main query in its clause. Such a reference, by a subquery to a field in its
enclosing query, is called an outer reference, and the corresponding
subquery is called a Correlated subquery, because its correlated with the
result set of one or more of the queries enclosing it.
When an outer reference appears within a subquery, MySQL has to run the
subquery once for every record generated by the outer query and, therefore,
test the subquery as many times as there are records in the outer querys
result set.
In such a context, the EXISTS operator comes in handy, to filter out certain
records from the final result set.
Example: Consider the following query, which reruns the previous example
with an outer reference to make the result set more useful:
If you look at the data, only one branch is using four or more services
(branch ID 1011).
In this case because the inner query contains a reference to a field in the
outer query, MySQL cannot run the inner query only once (as it usually
does). Rather, it has to run it over and over, once for every row in the outer
table, substitute the value of the named field from that row in the subquery,
and then decide whether to include that outer row in the final result set on
Sikkim Manipal University
Unit 6
the basis of whether the corresponding subquery returns a result set. This is
obviously expensive in terms of performance, and outer references should
be avoided unless absolutely necessary.
Self Assessment Questions
5. The ____ operator is most often used in the context of outer references.
The queries here are both TRUE if table t2 has a row where column1 = 1
and column2 = 2.
The expressions (1,2) and ROW(1,2) are sometimes called row
constructors. The two are equivalent. They are legal in other contexts as
well. For example, the following two statements are semantically equivalent
(although the first one cannot be optimized until MySQL 5.1.12):
SELECT * FROM t1 WHERE (column1,column2) = (1,1);
SELECT * FROM t1 WHERE column1 = 1 AND column2 = 1;
The normal use of row constructors is for comparisons with subqueries that
return two or more columns. For example, the following query answers the
request, find all rows in table t1 that also exist in table t2:
SELECT column1,column2,column3
FROM t1
WHERE (column1,column2,column3) IN
(SELECT column1,column2,column3 FROM t2);
Unit 6
The [AS] name clause is mandatory, because every table in a FROM clause
must have a name. Any columns in the subquery select list must have
unique names.
For the sake of illustration, assume that you have this table:
Here is how to use a subquery in the FROM clause, using the example
table:
Subqueries in the FROM clause can return a scalar, column, row, or table.
Subqueries in the FROM clause cannot be correlated subqueries, unless
used within the ON clause of a JOIN operation.
Sikkim Manipal University
Unit 6
6.10 Summary
A Subquery is exactly what it sounds like: a SELECT query that is
subordinate to another query. This unit covers the various concepts of
subqueries.
1. The subquery as scalar operand: A scalar subquery is a simple
operand, and you can use it almost anywhere a single column value or
literal is legal, and you can expect it to have those characteristics that all
operands have: a data type, a length, an indication whether it can be
NULL, and so on.
2. Comparisons Using Subqueries: The output of the subqueries can be
used to compare the output of the outer most queries. This can be done
using various comparison operators like >, <, IN, and so on.
3. Subqueries with ANY, IN, and SOME: These type of subqueries can
be used in cases where a set of values need to be compared and the
output determined.
4. Subqueries with ALL: These type of subqueries can be used in cases
where a set of values are the output generated by the inner or
subqueries and the output is possible only if all the values match with
the outer query.
5. Correlated Subqueries: Sometimes a situation arises in which a
subquery uses a field from the main query in its clause. Such a
reference by a subquery to a field in its enclosing query, is called an
outer reference, and the corresponding subquery is called a correlated
subquery, because its correlated with the result set of one or more of
the queries enclosing it.
6. EXISTS and NOT EXISTS: The Exists operator can be used to check if
a subquery produces any results at all. The NOT EXISTS operator is
exactly the opposite of the output produced by NOT EXISTS.
7. Row Subqueries: A row subquery is a subquery variant that returns a
single row and can thus return more than one column value.
8. Subqueries in the FROM Clause: Subqueries are legal in a SELECT
statement's FROM clause. Any columns in the subquery select list must
have unique names.
Sikkim Manipal University
Unit 6
6.12 Answers
Self Assessment Questions
1. scalar
2. parentheses
3. NOT IN
4. outer
5. EXISTS
6. row
Terminal Questions
1. (Refer Section 6.1)
They allow queries that are structured so that it is possible to isolate
each part of a statement.
They provide alternative ways to perform operations that would
otherwise require complex joins and unions.
They are, in many people's opinion, more readable than complex
joins or unions.
2. Sometimes a situation arises in which a subquery uses a field from the
main query in its clause. Such a reference by a subquery to a field in its
enclosing query, is called an outer reference, and the corresponding
subquery is called a correlated subquery, because its correlated with
the result set of one or more of the queries enclosing it.
When an outer reference appears within a subquery, MySQL has to run
the subquery once for every record generated by the outer query, and
therefore test the subquery as many times as there are records in the
outer querys result set. In such a context, the EXISTS operator comes
in handy, to filter out certain records from the final result set.
Unit 6
Example:
SELECT * FROM t1 WHERE column1 = ANY
(SELECT column1 FROM t2 WHERE t2.column2 =
t1.column2);
1 and column2 = 2.
The expressions (1,2) and ROW(1,2) are sometimes called row
constructors. The two are equivalent. They are legal in other contexts as
well.
Unit 7
Unit 7
Structure
7.1 Introduction
Objectives
7.2 Operators
Parentheses
Comparison Operators
Logical Operators
Case Sensitivity Operators
7.3 Control Flow Functions
7.4 String Functions
7.5 Numeric Functions
Arithmetic Operators
Mathematical Functions
7.6 Date and Time Functions
7.7 Summary
7.8 Terminal Questions
7.9 Answers
7.1 Introduction
We have already seen in the previous units, how MySQL statements such
as the SELECT statement can use comparison operators to retrieve only
those records matching a specified condition or a set of conditions. The
operators can be used to build complex expressions and clauses that allow
you to perform sophisticated comparison and conversion operations on the
data in your MySQL tables.
This unit will introduce you to MySQLs numerous operators, explaining the
usage of each using appropriate examples, perform calculations and
comparisons, create conditional groups, search for regular expressions in
strings, and data conversion methods.
The MySQL operators can be classified into the following four categories:
1. Arithmetic Operators
2. Comparison Operators
3. Logical Operators
4. Bit Operators
Sikkim Manipal University
Unit 7
Objectives
After studying this unit, you should be able to:
discuss the importance and applications of various operators
describe and apply various Operators like Parenthesis, Comparison, etc.
explain the importance of various control flow functions along with
applications
describe and apply various string functions
describe various Arithmetic operators and Mathematical Functions
explain the date and time functions with relevant examples
7.2 Operators
7.2.1 Parenthesis
( ... )
BETWEEN
IN
IS NULL
IS NOT NULL
LIKE
REGEXP aka RLIKE
Unit 7
The opposite of the = operator is the <> operator, which is used to test
whether the two sides of an expression are unequal; it returns true if they
are and false if they are not.
Sikkim Manipal University
Unit 7
The <> operator can also be used for comparison with NULL values:
Unit 7
You can reverse the results of a BETWEEN test by adding the NOT logical
operator; this returns true only if the expression lies outside the given range:
A comparison test using the IN operator returns true if the value being
tested exists in the specified set. This operator can be used both with strings
as well as numbers.
Unit 7
You can perform the case-sensitive comparison with the BINARY keyword:
Example:
To test for the presence or absence of actual NULL values in an expression,
we use the IS NULL and IS NOT NULL operators, which are designed
specifically to perform comparisons involving NULL values.
The following example demonstrates how the IS NULL operator can be
used to test whether or not a value is null:
Unit 7
The <= > operator, referred to as a NULL safe equality operator. If, in a
deviation from its normal behavior, MySQL returns a true or false result for
the comparison even when the expressions involved in the comparison
contain a NULL value.
The above three mentioned operators come in handy when dealing with
MySQL columns containing NULL data.
To perform wildcard searches of data, you should use the LIKE operator. It
allows the selection of records matching all or part of a specified string by
allowing the use of special wild card characters in an expression.
Example:
In this case, the % wildcard character tells MySQL to match all those values
zero or more occurrences of the substring ll; for example hello, ball, roller,
and so on.
Unit 7
The NOT Logical Operator: Adding this operator reverses the test,
returning those records that do not match the specified string.
Example - 2:
Unit 7
Addition of the NOT logical operator reverses the result, returning false if a
match is found.
The table below lists a set of useful meta characters along with the function
of each:
Table 7.2: Meta characters and their meanings
Meta Character
Function or Meaning
\s
\S
\d
\w
\W
Unit 7
Function or Meaning
NOT aka !
Logical NOT
Logical AND
OR aka ||
Logical OR
XOR
The NOT Operator: It reverses the logical sense of the test following it,
turning true into false and false into true.
The AND Operator: This operator makes it possible to test the validity of
two or more values (or expressions evaluating to values); it returns true if all
its components are true and not NULL, and it returns false otherwise.
Unit 7
Example -2:
Before MySQL 4.1.1, use the UPPER() function to convert the BLOB
value to uppercase before performing the comparison:
Example:
Unit 7
For MySQL 4.1.1 and up, BLOB columns have a character set of binary,
which has no concept of lettercase. To perform a case-insensitive
comparison, use the CONVERT() function to convert the BLOB value to
a character set that is not case sensitive. The result is a non-binary
string, so the LIKE operation is not case sensitive:
Example:
SELECT 'A' LIKE CONVERT(blob_col USING latin1) FROM
tbl_name;
To use a different character set, substitute its name for latin1 in the
preceding statement.
4. Bit Operators
MySQL has 6 operators specifically designed for bit manipulations. The
table below shows the list:
Table 7.4: Bit wise operators
Operator
Function or Meaning
&
Bitwise AND
Bitwise OR
Bit Inversion
>>
<<
The | operator is used to perform a bitwise OR, while the & operator is
used to perform a bitwise AND.
Unit 7
You can shift the bits to the left and right with the << AND >> operators,
respectively consider the example, at the top of the next page, which
demonstrates:
Unit 7
Unit 7
Example:
mysql> SELECT IF(1>2,2,3);
-> 3
mysql> SELECT IF(1<2,'yes','no');
-> 'yes'
mysql> SELECT IF(STRCMP('test','test1'),'no','yes');
-> 'no'
If only one of expr2 or expr3 is explicitly NULL, the result type of the IF()
function is the type of non-NULL expression.
expr1 is evaluated as an integer value, which means that if you are testing
floating-point or string values, you should do so using a comparison
operation.
Example:
Unit 7
4. NULLIF(expr1,expr2)
Returns NULL if expr1 = expr2 is true, else returns expr1. This is the
same as CASE WHEN x = y THEN NULL ELSE x END.
mysql> SELECT NULLIF(1,1);
-> NULL
mysql> SELECT NULLIF(1,2);
-> 1
Note: MySQL evaluates expr1 twice if the arguments are not equal.
Self Assessment Questions
3. A comparison test using the BETWEEN operator returns true if the
expression being tested lies between the specified range, ____ of both
end points of the range.
BIN(N)
Unit 7
BIT_LENGTH(str)
CHAR(N,...)
CHAR_LENGTH(str)
CONCAT(str1,str2,...)
INSERT(str,pos,len,
newstr)
SELECT
BIT_LENGTH('text');
-> 32
1.SELECT
CHAR(77,121,83,81,'76');
-> 'MySQL'
2.SELECT
CHAR(77,77.3,'77.3');
-> 'MMM'
1. SELECT CONCAT('My',
'S', 'QL');
-> 'MySQL'
2. SELECT CONCAT('My',
NULL, 'QL');
-> NULL
SELECT
INSERT('Quadratic', 3, 4,
'What');
-> 'QuWhattic'
INSTR(str,substr)
LEFT(str,len)
LENGTH(str)
LOCATE(substr,str)
LOCATE(substr,str,
pos)
LOWER(str)
Unit 7
1.SELECT
INSTR('foobarbar', 'bar');
-> 4
2. SELECT INSTR('xbar',
'foobar');
-> 0
SELECT LEFT('foobarbar',
5);
-> 'fooba'
SELECT LENGTH('text');
-> 4
1. SELECT LOCATE('bar',
'foobarbar');
-> 4
2. SELECT
LOCATE('xbar', 'foobar');
-> 0
SELECT
LOWER('QUADRATICALL
Y');
-> 'quadratically'
LPAD(str,len,padstr)
LTRIM(str)
REPEAT(str,count)
REPLACE(str,from_
str,to_str)
REVERSE(str)
RIGHT(str,len)
RPAD(str,len,padstr)
Unit 7
1. SELECT
LPAD('hi',4,'??');
-> '??hi'
2. SELECT
LPAD('hi',1,'??');
-> 'h'
SELECT
REPEAT('MySQL', 3);
-> 'MySQLMySQLMySQL'
SELECT
REPLACE('www.mysql.co
m', 'w', 'Ww');
-> 'WwWwWw.mysql.com'
SELECT REVERSE('abc');
-> 'cba'
SELECT
RIGHT('foobarbar', 4);
-> 'rbar'
SELECT RPAD('hi',5,'?');
-> 'hi???'
RTRIM(str)
SUBSTRING(str,pos)
SUBSTRING(str
FROM pos)
SUBSTRING(str,pos,l
en)
SUBSTRING(str
FROM pos FOR len)
UPPER(str)
Unit 7
SELECT UPPER('Hej');
-> 'HEJ'
1. SELECT
SUBSTRING('Quadraticall
y',5);
-> 'ratically'
2. SELECT
SUBSTRING('foobarbar'
FROM 4);
-> 'barbar'
3. SELECT
SUBSTRING('Quadraticall
y',5,6);
-> 'ratica'
Unit 7
Character
%
_
(Underscore)
Description
Matches any number of
characters, even zero
characters
Matches exactly one
character
Example
SELECT
'David!'
LIKE 'David_';
-> 1
SELECT
'David!'
LIKE '%D%v%';
-> 1
Unit 7
Example:
mysql> SELECT 'Monty!' REGEXP 'm%y%%';
-> 0
mysql> SELECT 'Monty!' REGEXP '.*';
-> 1
mysql> SELECT 'new*\n*line' REGEXP 'new\\*.\\*line';
-> 1
mysql> SELECT 'a' REGEXP 'A', 'a' REGEXP BINARY 'A';
-> 1 0
mysql> SELECT 'a' REGEXP '^[a-d]';
-> 1
3. STRCMP(expr1,expr2)
STRCMP() returns 0 if the strings are the same, -1 if the first argument is
smaller than the second according to the current sort order, and 1
otherwise.
Example:
mysql> SELECT STRCMP('text', 'text2');
-> -1
mysql> SELECT STRCMP('text2', 'text');
-> 1
mysql> SELECT STRCMP('text', 'text');
-> 0
Name
Description
DIV(v4.1.0)
/
%
+
*
Integer division
Division operator
Minus operator
Modulo operator
Addition operator
Times operator
Change the sign of the
argument
Unit 7
CONV()
COS()
COT()
CRC32()(v4.1.0)
DEGREES()
EXP()
FLOOR()
LN()
LOG10()
LOG2()
LOG()
MOD()
OCT()
PI()
POW()
POWER()
RADIANS()
RAND()
ROUND()
SIGN()
SIN()
SQRT()
TAN()
TRUNCATE()
Unit 7
Unit 7
Description
ADDDATE()(v4.1.1)
Add dates
ADDTIME()(v4.1.1)
Add time
CONVERT_TZ()(v4.1.3)
CURDATE()
CURRENT_DATE(), CURRENT_DATE
CURRENT_TIME(), CURRENT_TIME
CURRENT_TIMESTAMP(),
CURRENT_TIMESTAMP
CURTIME()
DATE_ADD()
DATE_FORMAT()
DATE_SUB()
DATE()(v4.1.1)
DATEDIFF()(v4.1.1)
DAY()(v4.1.1)
DAYNAME()(v4.1.21)
Unit 7
DAYOFMONTH()
DAYOFWEEK()
DAYOFYEAR()
EXTRACT
FROM_DAYS()
FROM_UNIXTIME()
GET_FORMAT()(v4.1.1)
HOUR()
LAST_DAY(v4.1.1)
LOCALTIME(), LOCALTIME
LOCALTIMESTAMP,
LOCALTIMESTAMP()(v4.0.6)
MAKEDATE()(v4.1.1)
MAKETIME(v4.1.1)
MAKETIME()
MICROSECOND()(v4.1.1)
Return
the
argument
MINUTE()
MONTH()
MONTHNAME()(v4.1.21)
NOW()
PERIOD_ADD()
PERIOD_DIFF()
QUARTER()
Return the
argument
SEC_TO_TIME()
Converts
format
SECOND()
STR_TO_DATE()(v4.1.1)
SUBDATE()
SUBTIME()(v4.1.1)
Subtract times
microseconds
quarter
seconds
from
to
from
date
'HH:MM:SS'
Unit 7
SYSDATE()
TIME_FORMAT()
Format as time
TIME_TO_SEC()
TIME()(v4.1.1)
TIMEDIFF()(v4.1.1)
Subtract time
TIMESTAMP()(v4.1.1)
TIMESTAMPADD()(v5.0.0)
Add an interval
expression
TIMESTAMPDIFF()(v5.0.0)
TO_DAYS()
UNIX_TIMESTAMP()
UTC_DATE()(v4.1.1)
UTC_TIME()(v4.1.1)
UTC_TIMESTAMP()(v4.1.1)
WEEK()
WEEKDAY()
WEEKOFYEAR()(v4.1.1)
YEAR()
YEARWEEK()
portion
to
of
the
datetime
Example: This uses date functions. The following query selects all rows
with a date_col value from within the last 30 days:
mysql> SELECT something FROM tbl_name
-> WHERE DATE_SUB(CURDATE(),INTERVAL 30 DAY) <= date_col;
The query also selects rows with dates that lie in the future.
Functions that expect date values usually accept datetime values and ignore
the time part. Functions that expect time values usually accept datetime
values and ignore the date part.
Sikkim Manipal University
Unit 7
Some date functions can be used with zero dates or incomplete dates
such as '2001-11-00', whereas others cannot. Functions that extract parts of
dates typically work with incomplete dates and thus can return 0 when you
might otherwise expect a non-zero value. For example:
mysql> SELECT DAYOFMONTH('2001-11-00'), MONTH('2005-00-00');
-> 0, 0
Other functions expect complete dates and return NULL for incomplete
dates. These include functions that perform date arithmetic or that map
parts of dates to names.
Example:
mysql> SELECT DATE_ADD('2006-05-00',INTERVAL 1 DAY);
-> NULL
mysql> SELECT DAYNAME('2006-05-00');
-> NULL
7.7 Summary
This topic covered various operators and functions used in SQL statements
as well as stored procedures and functions:
1. Operators: Operators are the objects used inside a query or a subquery
for performing various operations like precedence overriding,
comparisons, Logical operations and so on.
2. Control Flow Functions: These are the functions that control the flow
of operations in a procedure or a function within programs. They include
functions like CASE, IFELSE structures and so on.
3. String Functions: These are the functions that are used to manipulate
individual characters or a set of characters clubbed together as strings.
They include functions like ASCII, BIT, Length, Insert, and so on.
4. Numeric Functions: These are the functions that deal with arithmetic
operations like additions, subtractions, and so on. They even cover the
trigonometric and statistical functions.
Unit 7
5. Date and Time Functions: This topic discusses various functions that
can be used to manipulate the dates. These functions include date
arithmetic, manipulation and formatting.
7.9 Answers
Self Assessment Questions
1. parentheses
2. inclusive
3. numeric
4. 100
5. NULL
6. 'h'
7. 1
8. NULL
9. MAKEDATE
Terminal Questions
1. (Refer Section 7.2)
a. Parentheses: It is used to force the order of evaluation in an
expression.
b. Comparison Operators: These operators allow you to compare the
left side of an expression with its right side. The result of such a
comparison operator is always 1(TRUE) or 0 (FALSE) or NULL
(cannot be determined).
c. MySQL uses four logical operators, which makes it possible to test
the logical validity of one or more expressions (or sets of
expressions). The result of an operation involving these operators is
always 1(true), 0 (false), or NULL (could not be determined).
Unit 7
Function or Meaning
Logical NOT
Logical AND
Logical OR
Logical XOR (Exclusive OR)
INSERT(str,pos,len,newstr)
LOWER(str)
LTRIM(str)
SUBSTRING(str,pos)
SUBSTRING(str FROM
pos)
SUBSTRING(str,pos,len)
SUBSTRING(str FROM pos
FOR len)
Description
Example
1. SELECT
CONCAT('My', 'S', 'QL');
-> 'MySQL'
2. SELECT
CONCAT('My', NULL,
'QL');
-> NULL
SELECT
INSERT('Quadratic', 3,
4, 'What');
-> 'QuWhattic'
SELECT
LOWER('QUADRATICA
LLY');
-> 'quadratically'
SELECT LTRIM('
barbar');
-> 'barbar'
1. SELECT
SUBSTRING('Quadratic
ally',5);
-> 'ratically'
2. SELECT
SUBSTRING('foobarbar'
FROM 4);
-> 'barbar'
3. SELECT
SUBSTRING('Quadratic
ally',5,6);
-> 'ratica'
Unit 8
8.1
8.2
8.3
8.4
8.5
8.6
8.7
8.8
Unit 8
Advanced Functions
Introduction
Objectives
Full-text Search Functions
Boolean Full-text Searches
Full-text Searches with Query Expansion
Full-text Restrictions
Fine-tuning MySQL Full-text Search
Full-text Search TODO
Cast Functions
Other Functions
Bit Functions
Encryption Functions
Information Functions
Functions and Modifiers for Use with GROUP BY Clauses
GROUP BY (Aggregate) Functions
GROUP BY Modifiers
Summary
Terminal Questions
Answers
8.1 Introduction
This unit is an extension to the previous unit, where in different operators
and functions were discussed in detail. This unit exclusively discusses the
text based search functions. It also describes the Cast, Bit, Encryption and
Information functions. It also discusses in detail the set of functions and
modifiers that can be used in conjunction with GROUP BY modifier with
appropriate examples.
Objectives
After studying this unit, you should be able to:
explain and apply Full text search functions
describe various cast functions
discuss the importance of Bit, Encryption and Information functions
discuss the functions and modifiers for use with Group BY clauses
Unit 8
Unit 8
Unit 8
Unit 8
affected. Words match if they begin with the word preceding the *
operator.
If a stopword or too-short word is specified with the truncation
operator, it will not be stripped from a boolean query. For example, a
search for '+word +stopword*' will likely return fewer rows than a
search for '+word +stopword' because the former query remains as
is and requires stopword* to be present in a document. The latter
query is transformed to +word.
": A phrase that is enclosed within double quote (") characters
matches only rows that contain the phrase literally, as it was typed.
The full-text engine splits the phrase into words, performs a search
in the FULLTEXT index for the words. Non-word characters need not
be matched exactly: Phrase searching requires only that matches
contain exactly the same words as the phrase and in the same
order. For example, "test phrase" matches "test, phrase".
If the phrase contains no words that are in the index, the result is
empty. For example, if all words are either stopwords or shorter than
the minimum length of indexed words, the result is empty.
8.2.2 Full-text Searches with Query Expansion
Full-text search supports query expansion (and in particular, its variant
blind query expansion). This is generally useful when a search phrase is
too short, which often means that the user is relying on implied knowledge
that the full-text search engine lacks. For example, a user searching for
database may really mean that MySQL, Oracle, DB2, and RDBMS
all are phrases that should match databases and should be returned, too.
This is implied knowledge.
Blind query expansion (also known as automatic relevance feedback) is
enabled by adding WITH QUERY EXPANSION or IN NATURAL
LANGUAGE MODE WITH QUERY EXPANSION following the search
phrase. It works by performing the search twice, where the search phrase
for the second search is the original search phrase concatenated with the
few most highly relevant documents from the first search. Thus, if one of
these documents contains the word databases and the word MySQL, the
second search finds the documents that contain the word MySQL even if
they do not contain the word database. The following example shows this
difference:
Sikkim Manipal University
Unit 8
Unit 8
BOOLEAN MODE. Boolean-mode searches can be done on nonindexed columns, although they are likely to be slow.
The argument to AGAINST() must be a constant string.
8.2.4 Fine - tuning MySQL Full-text Search
MySQL's full-text search capability has few user-tunable parameters. You
can exert more control over full-text searching behavior if you have a
MySQL source distribution because some changes require source code
modifications..
Note that full-text search is carefully tuned for the most effectiveness.
Modifying the default behavior in most cases can actually decrease
effectiveness. Do not alter the MySQL sources unless you know what you
are doing.
Most full-text variables described in this section must be set at server
startup time. A server restart is required to change them; they cannot be
modified while the server is running.
Some variable changes require that you rebuild the FULLTEXT indexes in
your tables. Instructions for doing this are given at the end of this section.
The minimum and maximum lengths of words to be indexed are
defined by the ft_min_word_len and ft_max_word_len system
variables. The default minimum value is four characters; the default
maximum is version dependent. If you change either value, you must
rebuild your FULLTEXT indexes. For example, if you want threecharacter words to be searchable, you can set the ft_min_word_len
variable by putting the following lines in an option file:
[mysqld]
ft_min_word_len=3
Then you must restart the server and rebuild your FULLTEXT indexes. Note
particularly the remarks regarding myisamchk in the instructions following
this list.
To override the default stopword list, set the ft_stopword_file system
variable. The variable value should be the pathname of the file
containing the stopword list, or the empty string to disable stopword
filtering. After changing the value of this variable or the contents of
Unit 8
the stopword file, restart the server and rebuild your FULLTEXT
indexes.
The stopword list is free-form. That is, you may use any nonalphanumeric character such as newline, space, or comma to
separate stopwords. Exceptions are the underscore character (_)
and a single apostrophe (') which are treated as part of a word. The
character set of the stopword list is the server's default character set
The 50% threshold for natural language searches is determined by
the particular weighting scheme chosen. To disable it, look for the
following line in storage/myisam/ftdefs.h:
#define GWS_IN_USE
GWS_PROB
Unit 8
Make the model more flexible (by adding some adjustable parameters to
FULLTEXT in CREATE TABLE and ALTER TABLE statements).
Unit 8
The functions also can be useful for sorting ENUM columns in lexical order.
Normally sorting of ENUM columns occurs using the internal numeric
values. Casting the values to CHAR results in a lexical sort:
SELECT enum_col FROM tbl_name ORDER BY CAST(enum_col AS CHAR);
Unit 8
Unit 8
Unit 8
You can get even more security by not transferring the key over the
connection for each query, which can be accomplished by storing it in a
server-side variable at connection time.
Example:
SELECT @password:='my password';
INSERT INTO t VALUES (1,AES_ENCRYPT('text',@password));
Unit 8
Encrypts the string with the given key using the Triple-DES algorithm. On
error, this function returns NULL.
Note that this function works only if MySQL has been configured with SSL
support.
The encryption key to use is chosen based on the second argument to
DES_ENCRYPT(), if one was given:
SNo
Argument
Description
1
2
No argument
key_num
key_str
The key file can be specified with the des-key-file server option. The return
string is a binary string where the first character is CHAR(128 | key_num).
Unit 8
Each key_num must be a number in the range from 0 to 9. Lines in the file
may be in any order. des_key_str is the string that will be used to encrypt
the message. Between the number and the key there should be at least one
space.
The first key is the default key that is used if you dont specify any key
argument to DES_ENCRYPT()
You can tell MySQL to read new key values from the key file with the
FLUSH DES_KEY_FILE command. This requires the RELOAD privilege.
One benefit of having a set of default keys is that it gives applications a way
to check for the existence of encrypted column values, without giving the
end user the right to decrypt those values.
ENCRYPT(str[,salt])
Encrypt str using the Unix crypt() system call. The salt argument should be a
string with two characters. (As of MySQL 3.22.16, salt may be longer than
two characters.)
mysql> SELECT ENCRYPT('hello');
-> 'VxuFAJXVARROc'
ENCRYPT() ignores all but the first 8 characters of str, at least on some
systems. This behavior is determined by the implementation of the
underlying crypt() system call. If crypt() is not available on your system,
ENCRYPT() always returns NULL. Because of this, we recommend that you
use MD5() or SHA1() instead, because these two functions exist on all
platforms.
Unit 8
MD5(str)
Calculates an MD5 128-bit checksum for the string. The value is returned as
a string of 32 hex digits, or NULL if the argument was NULL. The return
value can, for example, be used as a hash key.
mysql> SELECT MD5('testing');
-> 'ae2b1fca515949e5d54fb22b8ed95575'
Unit 8
The time reported is elapsed time on the client end, not CPU time on the
server end. It may be advisable to execute BENCHMARK() several times,
and interpret the result with regard to how heavily loaded the server
machine is.
CHARSET(str)
Returns the character set of the string argument.
Unit 8
Unit 8
CONNECTION_ID()
Returns the connection ID (thread ID) for the connection. Every connection
has its own unique ID.
CURRENT_USER()
Returns the username and hostname combination that the current session
was authenticated as. This value corresponds to the account that is used for
assessing your access privileges. It may be different than the value of
USER().
Unit 8
DATABASE()
Returns the default (current) database name.
The second SELECT will return a number indicating how many rows the first
SELECT would have returned had it been written without the LIMIT clause.
(If the preceding SELECT statement does not include the
SQL_CALC_FOUND_ROWS option, then FOUND_ROWS() may return a
different result when LIMIT is used than when it is not.)
Note: If you are using SELECT SQL_CALC_FOUND_ROWS, MySQL must
calculate how many rows are in the full result set. However, this is faster
than running the query again without LIMIT, because the result set need not
be sent to the client.
Unit 8
The last ID that was generated is maintained in the server on a perconnection basis. This means the value the function returns to a given client
is the most recent AUTO_INCREMENT value generated by that client. The
value cannot be affected by other clients, even if they generate
AUTO_INCREMENT values of their own. This behavior ensures that you
can retrieve your own ID without concern for the activity of other clients, and
without the need for locks or transactions.
The value of LAST_INSERT_ID() is not changed if you update the
AUTO_INCREMENT column of a row with a non-magic value (that is, a
value that is not NULL and not 0). If you insert many rows at the same time
with an insert statement, LAST_INSERT_ID() returns the value for the first
inserted row. The reason for this is to make it possible to easily reproduce
the same INSERT statement against some other server.
If expr is given as an argument to LAST_INSERT_ID(), then the value of the
argument is returned by the function, and is set as the next value to be
returned by LAST_INSERT_ID(). This can be used to simulate sequences:
Sikkim Manipal University
Unit 8
The UPDATE statement increments the sequence counter and causes the
next call to LAST_INSERT_ID() to return the updated value. The SELECT
statement retrieves that value.
VERSION()
Returns a string that indicates the MySQL server version.
Note: If your version string ends with -log this means that logging is enabled
8.4.4 Miscellaneous Functions
FORMAT(X,D)
Formats the number X to a format like '#,###,###.##', rounded to D
decimals, and returns the result as a string. If D is 0, the result will have no
decimal point or fractional part.
Unit 8
Example 1:
Example 2:
Example 3:
Unit 8
AVG([DISTINCT] expr)
Returns the average value of expr. The DISTINCT option can be used as of
MySQL 5.0.3 to return the average of the distinct values of expr.
AVG() returns NULL if there were no matching rows.
mysql> SELECT student_name, AVG(test_score)
->
FROM student
->
GROUP BY student_name;
BIT_AND(expr)
Returns the bitwise AND of all bits in expr. The calculation is performed with
64-bit (BIGINT) precision.
This function returns 18446744073709551615 if there were no matching
rows. (This is the value of an unsigned BIGINT value with all bits set to 1.)
BIT_OR(expr)
Returns the bitwise OR of all bits in expr. The calculation is performed with
64-bit (BIGINT) precision.
This function returns 0 if there were no matching rows.
BIT_XOR(expr)
Returns the bitwise XOR of all bits in expr. The calculation is performed with
64-bit (BIGINT) precision.
This function returns 0 if there were no matching rows.
Sikkim Manipal University
Unit 8
COUNT(expr)
Returns a count of the number of non-NULL values in the rows retrieved by
a SELECT statement.
Example:
mysql> SELECT student.student_name,COUNT(*)
-> FROM student,course
-> WHERE student.student_id=course.student_id
-> GROUP BY student_name;
GROUP_CONCAT(expr)
This function returns a string result with the concatenated values from a
group.
The full syntax is as follows:
GROUP_CONCAT([DISTINCT] expr [,expr ...]
[ORDER BY {unsigned_integer | col_name | expr}
[ASC | DESC] [,col ...]]
[SEPARATOR str_val])
OR
Unit 8
MIN(expr)
MAX(expr)
Returns the minimum or maximum value of expr. MIN() and MAX() may
take a string argument; in such cases they return the minimum or
maximum string value.
Example:
mysql> SELECT student_name, MIN(test_score), MAX(test_score)
-> FROM student
-> GROUP BY student_name;
STD(expr)
STDDEV(expr)
Returns the standard deviation of expr (the square root of VARIANCE()).
This is an extension to standard SQL. The STDDEV() form of this
function is provided for Oracle compatibility.
SUM(expr)
Returns the sum of expr. Note that if the return set has no rows, it returns
NULL!
VARIANCE(expr)
Returns the standard variance of expr (considering rows as the whole
population, not as a sample; so it has the number of rows as denominator).
This is an extension to standard SQL, available only in MySQL 4.1 or later.
8.5.2 GROUP BY Modifiers
As of MySQL 4.1.1, the GROUP BY clause allows a WITH ROLLUP
modifier that causes extra rows to be added to the summary output. These
rows represent higher-level (or superaggregate) summary operations.
ROLLUP thus allows you to answer questions at multiple levels of analysis
Sikkim Manipal University
Unit 8
with a single query. It can be used, for example, to provide support for
OLAP (Online Analytical Processing) operations.
As an illustration, suppose that a table named sales has year, country,
product, and profit columns for recording sales profitability:
CREATE TABLE sales
(
year INT NOT NULL,
country VARCHAR(20) NOT NULL,
product VARCHAR(32) NOT NULL,
profit INT
);
The tables contents can be summarized per year with a simple GROUP BY
like this:
mysql> SELECT year, SUM(profit) FROM sales GROUP BY year;
+------+-------------+
| year | SUM(profit) |
+------+-------------+
| 2000 | 4525
|
| 2001 | 3010
|
+------+-------------+
This output shows the total profit for each year, but if you also want to
determine the total profit summed over all years, you must add up the
individual values yourself or run an additional query.
Or you can use ROLLUP, which provides both levels of analysis with a
single query. Adding a WITH ROLLUP modifier to the GROUP BY clause
causes the query to produce another row that shows the grand total over all
year values:
mysql> SELECT year, SUM(profit) FROM sales GROUP BY year
WITH ROLLUP;
+------+-------------+
| year | SUM(profit) |
+------+-------------+
| 2000 | 4525 |
| 2001 | 3010 |
| NULL | 7535 |
+------+-------------+
Unit 8
The grand total super-aggregate line is identified by the value NULL in the
year column.
ROLLUP has a more complex effect when there are multiple GROUP BY
columns. In this case, each time there is a break (change in value) in any
but the last grouping column, the query produces an extra super-aggregate
summary row.
For example, without ROLLUP, a summary on the sales table based on
year, country, and product might look like this:
mysql> SELECT year, country, product, SUM(profit)
-> FROM sales
-> GROUP BY year, country, product;
+------+---------+------------+-------------+
| year | country | product | SUM(profit) |
+------+---------+------------+-------------+
| 2000 | Finland | Computer | 1500 |
| 2000 | Finland | Phone | 100 |
| 2000 | India | Calculator | 150 |
+------+---------+------------+-------------+
For this query, adding ROLLUP causes the output to include summary
information at four levels of analysis, not just one. Heres how to interpret
the ROLLUP output:
Unit 8
Following each set of product rows for a given year and country, an
extra summary row is produced showing the total for all products.
These rows have the product column set to NULL.
Following each set of rows for a given year, an extra summary row is
produced showing the total for all countries and products. These
rows have the country and products columns set to NULL.
Finally, following all other rows, an extra summary row is produced
showing the grand total for all years, countries, and products. This
row has the year, country, and products columns set to NULL.
8.6 Summary
This unit covered various text based search functions, casting functions, bit,
encryption and information functions. It then deals with various functions
and modifiers that can be used along with the GROUP BY clause.
1. Full text Search Functions: These functions are used to perform full
text search or string based functions. These functions include Boolean
searches, query expansion searches, and so on.
2. Cast Functions: The CAST() and CONVERT() functions may be used
to take a value of one type and produce a value of another type. These
functions are mainly used for performing conversions between different
data types.
3. Other Functions: The other functions include Bit, Encryption and
Information functions. These are used to work on Bit manipulations,
Encryption of data, and Information gathering from the data sources.
4. Group by Clauses: The Group By clause is basically used in
conjunction with the Aggregate functions like Sum, Average, and so on.
They may also be used to modify the output of a query depending on the
requirement.
Unit 8
8.8 Answers
Self Assessment Questions
1. FULLTEXT
2. CAST() and CONVERT()
3. 13
4. Non - NULL value
5. Temporal
Terminal Questions
1. (Refer to Section 8.2)
+: A leading plus sign indicates that this word must be present in each row
that is returned.
-: A leading minus sign indicates that this word must not be present in any of
the rows that are returned.
(no operator): By default (when neither + nor - is specified) the word is
optional, but the rows that contain it are rated higher. This mimics the
behavior of MATCH() ... AGAINST() without the IN BOOLEAN MODE
modifier.
> <: These two operators are used to change a word's contribution to the
relevance value that is assigned to a row. The > operator increases the
contribution and the < operator decreases it.
( ): Parentheses group words into subexpressions. Parenthesized groups
can be nested.
~: A leading tilde acts as a negation operator, causing the word's
contribution to the row's relevance to be negative.
2. (Refer to Section 8.4.2)
- AES_ENCRYPT(str,key_str)
- AES_DECRYPT(str,key_str)
These functions allow encryption/decryption of data using the official AES
(Advanced Encryption Standard) algorithm, previously known as Rijndael.
Encoding with a 128-bit key length is used, but you can extend it up to 256
bits by modifying the source. We chose 128 bits because it is much faster
Unit 8
and it is usually secure enough. The input arguments may be any length. If
either argument is NULL, the result of this function is also NULL.
DECODE(crypt_str,pass_str): Descrypts the encrypted string crypt_str
using pass_str as the password. crypt_str should be a string returned from
ENCODE().
ENCODE(str,pass_str): Encrypt str using pass_str as the password. To
decrypt the result, use DECODE(). The result is a binary string of the same
length as string. If you want to save it in a column, use a BLOB column type.
DES_DECRYPT(str_to_decrypt [, key_str]): Decrypts a string encrypted
with DES_ENCRYPT(). On error, this function returns NULL. Note that this
function works only if MySQL has been configured with SSL support.
Unit 9
Unit 9
Transaction Management
Structure
9.1
Introduction
Objectives
9.2
START TRANSACTION, COMMIT, and ROLLBACK Syntax
9.3
Statements That Cannot Be Rolled Back
9.4
Statements That Cause an Implicit Commit
9.5
SAVEPOINT and ROLLBACK TO SAVEPOINT Syntax
9.6
LOCK TABLES and UNLOCK TABLES Syntax
9.7
SET TRANSACTION Syntax
9.8
Summary
9.9
Terminal Questions
9.10 Answers
9.1 Introduction
This unit discusses about various transaction processing statements like
Start Transaction, Commit, and Rollback. It describes the statements that
cannot be rolled back after execution of a transaction. It also describes the
statements that cause an implicit commit after execution.
It discusses about the usage of Savepoints within a transaction and their
affect or usage during transaction processing.
Objectives
After studying this unit, you should be able to:
discuss the importance of Start transaction, commit and rollback
statements
describe the statements that cannot rollback
describe the statements that cause an implicit commit with relevant
examples
explain the importance of the statements Savepoint and Rollback to
Savepoint
demonstrate the process involved in Locking and unlocking the tables
with examples
explain the usage of Set Transaction syntax
Unit 9
Unit 9
Unit 9
log. Transactions are stored in the binary log in one chunk, upon COMMIT.
Transactions that are rolled back are not logged.
(Exception: Modifications to non-transactional tables cannot be rolled back.
If a transaction that is rolled back includes modifications to non-transactional
tables, the entire transaction is logged with a ROLLBACK statement at the
end to ensure that modifications to the non-transactional tables are
replicated.)
You can change the isolation level
TRANSACTION ISOLATION LEVEL.
for
transactions
with
SET
ALTER TABLE
BEGIN
CREATE INDEX
DROP DATABASE
DROP INDEX
DROP TABLE
LOAD MASTER DATA
LOCK TABLES
RENAME TABLE
Unit 9
SET AUTOCOMMIT=1
START TRANSACTION
TRUNCATE TABLE
UNLOCK TABLES also ends a transaction if any tables currently are locked.
Prior to MySQL 4.0.13, CREATE TABLE ends a transaction if the binary
update log is enabled.
Transactions cannot be nested. This is a consequence of the implicit
COMMIT performed for any current transaction when you issue a START
TRANSACTION statement or one of its synonyms.
ALTER TABLE, CREATE TABLE, and DROP TABLE do not commit a
transaction if the TEMPORARY keyword is used. (This does not apply to
other operations on temporary tables such as CREATE INDEX, which do
cause a commit.).
However, although no implicit commit occurs, neither can the statement be
rolled back. Therefore, use of such statements will violate transaction
atomicity: For example, if you use CREATE TEMPORARY TABLE and then
roll back the transaction, the table remains in existence.
The CREATE TABLE statement in InnoDB is processed as a single
transaction. This means that a ROLLBACK from the user does not undo
CREATE TABLE statements the user made during that transaction.
Beginning with MySQL 5.1.3, ALTER VIEW, CREATE TRIGGER, CREATE
VIEW, DROP TRIGGER, and DROP VIEW cause an implicit commit.
Beginning with MySQL 5.1.15, CREATE TABLE ... SELECT causes an
implicit commit before and after the statement is executed when you are
creating non-temporary tables. (No commit occurs for CREATE
TEMPORARY TABLE ... SELECT.)
This is to prevent an issue during replication where the table could be
created on the master after a rollback, but fail to be recorded in the binary
log, and therefore not replicated to the slave.
Statements that implicitly use or modify tables in the mysql
database: Beginning with MySQL 5.1.3, CREATE USER, DROP USER,
and RENAME USER cause an implicit commit. Beginning with MySQL
Sikkim Manipal University
Unit 9
Unit 9
Starting from MySQL 4.0.14 and 4.1.1, InnoDB supports the SQL
statements SAVEPOINT and ROLLBACK TO SAVEPOINT.
The SAVEPOINT statement sets a named transaction savepoint with a
name of identifier. If the current transaction already has a savepoint with the
same name, the old savepoint is deleted and a new one is set.
The ROLLBACK TO SAVEPOINT statement rolls back a transaction to the
named savepoint. Modifications that the current transaction made to rows
after the savepoint was set are undone in the rollback, but InnoDB does not
release the row locks that were stored in memory after the savepoint. (Note
that for a new inserted row, the lock information is carried by the transaction
ID stored in the row; the lock is not separately stored in memory. In this
case, the row lock is released in the undo.) Savepoints that were set at a
later time than the named savepoint are deleted.
If the statement returns the following error, it means that no savepoint with
the specified name exists:
ERROR 1181: Got error 153 during ROLLBACK
All savepoints of the current transaction are deleted if you execute a
COMMIT, or a ROLLBACK that does not name a savepoint.
Self Assessment Questions
3. The ______ statement sets a named transaction savepoint with a name
of identifier.
LOCK TABLES locks tables for the current thread. UNLOCK TABLES
releases any locks held by the current thread. All tables that are locked by
the current thread are implicitly unlocked when the thread issues another
LOCK TABLES, or when the connection to the server is closed.
Note: LOCK TABLES is not transaction-safe and implicitly commits any
active transactions before attempting to lock the tables.
Unit 9
As of MySQL 4.0.2, to use LOCK TABLES you must have the global LOCK
TABLES privilege and a SELECT privilege for the involved tables. In MySQL
3.23, you must have SELECT, INSERT, DELETE, and UPDATE privileges
for the tables.
The main reasons to use LOCK TABLES are for emulating transactions or
to get more speed when updating tables.
If a thread obtains a READ lock on a table, that thread (and all other
threads) can only read from the table. If a thread obtains a WRITE lock on a
table, only the thread holding the lock can read from or write to the table.
Other threads are blocked.
The difference between READ LOCAL and READ is that READ LOCAL
allows non-conflicting INSERT statements to execute while the lock is held.
However, this cant be used if you are going to manipulate the database files
outside MySQL while you hold the lock.
When you use LOCK TABLES, you must lock all tables that you are going to
use in your queries.
If you are using a table multiple times in a query (with aliases), you must get
a lock for each alias. While the locks obtained with a LOCK TABLES
statement are in effect, you cannot access any tables that were not locked
by the statement.
If your queries refer to a table using an alias, then you must lock the table
using that same alias. It will not work to lock the table without specifying the
alias:
mysql> LOCK TABLE t READ;
mysql> SELECT * FROM t AS myalias;
ERROR 1100: Table 'myalias' was not locked with LOCK TABLES
Conversely, if you lock a table using an alias, you must refer to it in your
queries using that alias:
mysql> LOCK TABLE t AS myalias READ;
mysql> SELECT * FROM t;
ERROR 1100: Table 't' was not locked with LOCK TABLES
mysql> SELECT * FROM t AS myalias;
Unit 9
WRITE locks normally have higher priority than READ locks, to ensure that
updates are processed as soon as possible. This means that if one thread
obtains a READ lock and then another thread requests a WRITE lock,
subsequent READ lock requests will wait until the WRITE thread has gotten
the lock and released it. You can use LOW_PRIORITY WRITE locks to
allow other threads to obtain READ locks while the thread is waiting for the
WRITE lock.
You should use LOW_PRIORITY WRITE locks only if you are sure that
there will eventually be a time when no threads will have a READ lock.
LOCK TABLES works as follows:
1. Sort all tables to be locked in an internally defined order. From the user
standpoint, this order is undefined.
2. If a table is locked with a read and a write lock, put the write lock before
the read lock.
3. Lock one table at a time until the thread gets all locks.
If you are going to run many operations on a set of tables, its much
faster to lock the tables you are going to use. The downside is that no
thread can update a READ-locked table (including the one holding the
lock) and no thread can read a WRITE-locked table other than the one
holding the lock.
The reason some operations are faster under LOCK TABLES is that
MySQL will not flush the key cache for the locked tables until UNLOCK
TABLES is called. Normally, the key cache is flushed after each SQL
statement. Locking MyISAM tables speeds up inserting, updating, or
deleting on them.
Unit 9
Without LOCK TABLES, it is possible that another thread might insert a new
row in the trans table between execution of the SELECT and UPDATE
statements.
You can avoid using LOCK TABLES in many cases by using relative
updates (UPDATE customer SET value=value+new_value) or the
LAST_INSERT_ID() function.
You can also avoid locking tables in some cases by using the user-level
advisory lock functions GET_LOCK() and RELEASE_LOCK(). These locks
are saved in a hash table in the server and implemented with
pthread_mutex_lock() and pthread_mutex_unlock() for high speed.
You can lock all tables in all databases with read locks with the FLUSH
TABLES WITH READ
LOCK statement. This is a very convenient way to get backups if you have a
file system such as Veritas that can take snapshots in time.
NOTE: If you use ALTER TABLE on a locked table, it may become
unlocked.
Self Assessment Questions
4. The ______ command releases any locks held by the current thread.
5. If a table is locked with a read and a write lock, put the ___ lock before
the ____ lock.
This statement sets the transaction isolation level for the next transaction,
globally, or for the current session.
The default behavior of SET TRANSACTION is to set the isolation level for
the next (not yet started) transaction.
Unit 9
If you use the GLOBAL keyword, the statement sets the default transaction
level globally for all new connections created from that point on. Existing
connections are unaffected. You need the SUPER privilege to do this.
Using the SESSION keyword sets the default transaction level for all future
transactions performed on the current connection.
You can set the initial default global isolation level for mysqld with the -transaction- isolation option.
To set the global default isolation level at server startup, use the -transaction-isolation=level option to mysqld on the command line or in an
option file.
Values of level for this option use dashes rather than spaces, so the
allowable values are READUNCOMMITTED, READ-COMMITTED,
REPEATABLE-READ, or SERIALIZABLE.
For example, to set the default isolation level to REPEATABLE READ, use
these lines in the [mysqld] section of an option file:
[mysqld]
transaction-isolation = REPEATABLE-READ
The following list describes how MySQL supports the different transaction
levels:
READ UNCOMMITTED
SELECT statements are performed in a non-locking fashion, but a
possible earlier version of a row might be used. Thus, using this isolation
level, such reads are not consistent. This is also called a dirty read.
Otherwise, this isolation level works like READ COMMITTED.
READ COMMITTED
Unit 9
A somewhat Oracle-like isolation level with respect to consistent (nonlocking) reads: Each consistent read, even within the same transaction,
sets and reads its own fresh snapshot.
For locking reads (SELECT with FOR UPDATE or LOCK IN SHARE
MODE), InnoDB locks only index records, not the gaps before them, and
thus allows the free insertion of new records next to locked records. For
UPDATE and DELETE statements, locking depends on whether the
statement uses a unique index with a unique search condition (such as
WHERE id = 100), or a range type search condition (such as WHERE id
> 100). For a unique index with a unique search condition, InnoDB locks
only the index record found, not the gap before it. For range-type
searches, InnoDB locks the index range scanned, using gap locks or
nextkey (gap plus index-record) locks to block insertions by other
sessions into the gaps covered by the range. This is necessary because
phantom rows must be blocked for MySQL replication and recovery to
work.
REPEATABLE READ
This is the default isolation level for InnoDB. For consistent reads, there
is an important difference from the READ COMMITTED isolation level:
All consistent reads within the same transaction read the snapshot
established by the first read. This convention means that if you issue
several plain (non-locking) SELECT statements within the same
transaction, these SELECT statements are consistent also with respect
to each other.
For locking reads (SELECT with FOR UPDATE or LOCK IN SHARE
MODE), UPDATE, and DELETE statements, locking depends on
whether the statement uses a unique index with a unique search
condition, or a range-type search condition. For a unique index with a
unique search condition, InnoDB locks only the index record found, not
the gap before it. For other search conditions, InnoDB locks the index
range scanned, using gap locks or next-key (gap plus index-record)
locks to block insertions by other sessions into the gaps covered by the
range.
SERIALIZABLE
Unit 9
This level is like REPEATABLE READ, but InnoDB implicitly converts all
plain SELECT statements to SELECT ... LOCK IN SHARE MODE if
autocommit is disabled. If autocommit is enabled, the SELECT is its own
transaction. It therefore is known to be read only and can be serialized if
performed as a consistent (non-locking) read and need not block for
other transactions. (This means that to force a plain SELECT to block if
other transactions have modified the selected rows, you should disable
autocommit.)
Self Assessment Questions
6. Using the _____ keyword sets the default transaction level for all future
transactions performed on the current connection.
9.8 Summary
This unit covers the transaction processing aspects of MySQL.
1. Start Transaction, Commit, and Rollback Syntax: This topic gives the
syntax and examples of Starting a transaction, committing the changes
done during a transaction, and rolling back any unwanted changes done
during a transaction.
2. Statements that cannot be rolled back: This topic discusses the
theory behind taking care of the transactions that cannot be rolled back.
3. Statements that cause an implicit Commit: This topic gives a list of
SQL statements that cause an implicit commit to the databases.
4. Savepoint and Rollback to Savepoint Syntax: The SAVEPOINT
statement sets a named transaction savepoint with a name of identifier.
The ROLLBACK TO SAVEPOINT statement rolls back a transaction to
the named savepoint.
5. Lock Tables and Unlock Tables Syntax: LOCK TABLES locks tables
for the current thread. UNLOCK TABLES releases any locks held by the
current thread. This topic also gives suitable examples for concept
understanding.
6. Set Transaction Syntax: This statement sets the transaction isolation
level for the next transaction, globally, or for the current session. The
default behavior of SET TRANSACTION is to set the isolation level for
the next (not yet started) transaction.
Sikkim Manipal University
Unit 9
9.10 Answers
Self Assessment Questions
1. disabled
2. UNLOCK TABLES
3. SAVEPOINT
4. UNLOCK TABLES
5. Write, Read
6. SESSION
Terminal Questions
1. (Refer Section 9.4)
The following statements implicitly end a transaction (as if you had done a
COMMIT before executing the statement):
UNLOCK TABLES also ends a transaction if any tables currently are locked.
Prior to MySQL 4.0.13, CREATE TABLE ends a transaction if the binary
update log is enabled.
Transactions cannot be nested. This is a consequence of the implicit
COMMIT performed for any current transaction when you issue a START
TRANSACTION statement or one of its synonyms.
ALTER TABLE, CREATE TABLE, and DROP TABLE do not commit a
transaction if the TEMPORARY keyword is used. (This does not apply to
other operations on temporary tables such as CREATE INDEX, which do
cause a commit.)
However, although no implicit commit occurs, neither can the statement be
rolled back. Therefore, use of such statements will violate transaction
atomicity: For example, if you use CREATE TEMPORARY TABLE and then
roll back the transaction, the table remains in existence.
Sikkim Manipal University
Unit 9
Starting from MySQL 4.0.14 and 4.1.1, InnoDB supports the SQL
statements SAVEPOINT and ROLLBACK TO SAVEPOINT.
The SAVEPOINT statement sets a named transaction savepoint with a
name of identifier. If the current transaction already has a savepoint with the
same name, the old savepoint is deleted and a new one is set.
The ROLLBACK TO SAVEPOINT statement rolls back a transaction to the
named savepoint. Modifications that the current transaction made to rows
after the savepoint was set are undone in the rollback, but InnoDB does not
release the row locks that were stored in memory after the savepoint. (Note
that for a new inserted row, the lock information is carried by the transaction
ID stored in the row; the lock is not separately stored in memory. In this
case, the row lock is released in the undo.) Savepoints that were set at a
later time than the named savepoint are deleted.
Unit 10
Unit 10
Stored Procedures
Structure
10.1 Introduction
Objectives
10.2 Stored Procedure and Routines
10.3 Maintaining Stored Procedures
CREATE PROCEDURE and CREATE FUNCTION
ALTER PROCEDURE and ALTER FUNCTION
DROP PROCEDURE and DROP FUNCTION
SHOW CREATE PROCEDURE and SHOW CREATE
FUNCTION
10.4 SHOW PROCEDURE STATUS and SHOW FUNCTION STATUS
10.5 CALL Statement
10.6 Summary
10.7 Terminal Questions
10.8 Answers
10.1 Introduction
This unit discusses stored programs and views, which are database objects
defined in terms of SQL code that is stored on the server for later invocation.
The stored procedures themselves can be written using PL/SQL. Stored
procedures exist to perform data-intensive operations that cannot be
accomplished using just SQL, and they perform these operations inside the
database where network performance is a moot issue.
Objectives
After studying this unit, you should be able to:
define and Distinguish between Stored Procedures and Routines
explain with examples the syntaxes of Create Procedure and Create
Function
explain with examples the syntaxes of Alter Procedure and Alter
Function
describe with examples the syntaxes of Drop Procedure and Drop
Function
Unit 10
Unit 10
If you use the mysql client program to define a stored program that contains
the semicolon characters within its definition, a problem arises. By default,
mysql itself recognizes semicolon as a statement delimiter, so you must
redefine the delimiter temporarily to cause mysql to pass the entire stored
program definition to the server.
To redefine the mysql delimiter, use the delimiter command. The following
example shows how to do this for the dorepeat() procedure just shown. The
delimiter is changed to // to enable the entire definition to be passed to the
server as a single statement, and then restored to ; before invoking the
procedure. This allows the; delimiter used in the procedure body to be
passed through to the server rather than being interpreted by mysql itself.
mysql> delimiter //
mysql> CREATE PROCEDURE dorepeat(p1 INT)
-> BEGIN
-> SET @x = 0;
-> REPEAT SET @x = @x + 1; UNTIL @x > p1 END REPEAT;
-> END
-> //
Query OK, 0 rows affected (0.00 sec)
mysql> delimiter ;
Unit 10
You can redefine the delimiter to a string other than //, and the delimiter can
consist of a single character or multiple characters. You should avoid the
use of the backslash (\) character because that is the escape character for
MySQL.
The following is an example of a function that takes a parameter, performs
an operation using an SQL function, and returns the result. In this case, it is
unnecessary to use delimiter because the function definition contains no
internal; statement delimiters:
Unit 10
Once this has been done, clients don't need to keep reissuing the individual
statements but can refer to the stored routine instead.
Stored routines require the proc table in the mysql database. This table is
created during the MySQL 5.1 installation procedure. If you are upgrading to
MySQL 5.1 from an earlier version, be sure to update your grant tables to
make sure that the proc table exists.
Stored routines can be particularly useful in certain situations:
When multiple client applications are written in different languages or
work on different platforms, but need to perform the same database
operations.
When security is paramount. Banks, for example, use stored procedures
and functions for all common operations. This provides a consistent and
secure environment, and routines can ensure that each operation is
properly logged. In such a setup, applications and users would have no
access to the database tables directly, but can only execute specific
stored routines.
Stored routines can provide improved performance because less
information needs to be sent between the server and the client. The tradeoff
is that this does increase the load on the database server because more of
the work is done on the server side and less is done on the client
(application) side. Consider this if many client machines (such as Web
servers) are serviced by only one or a few database servers.
Stored routines also allow you to have libraries of functions in the database
server. This is a feature shared by modern application languages that allow
such design internally (for example, by using classes). Using these client
application language features is beneficial for the programmer even outside
the scope of database use.
MySQL follows the SQL:2003 syntax for stored routines, which is also used
by IBM's DB2.
The MySQL implementation of stored routines is still in progress. All syntax
described here is supported and any limitations and extensions are
documented where appropriate.
Unit 10
func_parameter:
param_name type
type:
Any valid MySQL data type
characteristic:
LANGUAGE SQL
| [NOT] DETERMINISTIC
| {CONTAINS SQL | NO SQL | READS SQL DATA | MODIFIES SQL DATA }
| SQL SECURITY {DEFINER | INVOKER }
| COMMENT 'string'
routine_body:
Valid SQL procedure statement
Unit 10
This statement is used to drop a stored procedure or function. That is, the
specified routine is removed from the server. You must have the ALTER
ROUTINE privilege for the routine. (That privilege is granted automatically to
the routine creator.)
The IF EXISTS clause is a MySQL extension. It prevents an error from
occurring if the procedure or function does not exist.
DROP FUNCTION is also used to drop user-defined functions.
Sikkim Manipal University
Unit 10
This statement is a MySQL extension. It returns the exact string that can be
used to re-create the named stored procedure.
SHOW CREATE FUNCTION Syntax:
SHOW CREATE FUNCTION func_name
Unit 10
Unit 10
After the preceding code executes, row[0] and row[1] contain the values of
@param1 and @param2, respectively.
To handle INOUT parameters, execute a statement prior to the CALL that
sets the user variables to the values to be passed to the procedure.
Self Assessment Questions
5. The ___ statement can pass back values to its caller using parameters
that are declared as OUT or INOUT parameters.
6. To handle _____ parameters, execute a statement prior to the CALL that
sets the user variables to the values to be passed to the procedure.
Unit 10
10.6 Summary
Stored procedures exist to perform data-intensive operations that cannot be
accomplished using just SQL, and they perform these operations inside the
database where network performance is a moot issue
1. Stored Procedure and Routines: A Stored Procedure actually refers
collectively to standalone stored functions, standalone procedures,
packaged functions, and procedures.
To call a stored procedure, you need to know the procedure name and
know about any parameters that will be passed to the procedure. In the
case of a function, you also need to know the return type.
2. Maintaining Stored Procedures: This topic deals with the discussion of
functions like Create, Alter and Drop statements which allow the user to
do the said operations on procedures and functions. It also discusses
the Show syntaxes of Procedures and Functions to create them.
3. Show Procedure Status and Show Function Status: These
statements cause the output to display the metadata regarding the
procedures and functions created.
4. CALL Statement: The CALL statement invokes a stored procedure that
was defined previously with CREATE PROCEDURE. CALL can pass
back values to its caller using parameters that are declared as OUT or
INOUT parameters.
10.8 Answers
Self Assessment Questions
1. Stored Procedure
2. trigger
3. delimiter
Sikkim Manipal University
Unit 10
Unit 10
c.
Unit 11
Unit 11
Control Statements
Structure
11.1 Introduction
Objectives
11.2 BEGIN ... END Compound Statement
11.3 DECLARE Statement
11.4 Variables in Stored Procedures
11.5 Variable SET Statement
11.6 SELECT ... INTO Statement
11.7 DECLARE Conditions and Handlers
11.8 Flow Control Constructs
IF Statement
CASE Statement
LOOP Statement
LEAVE Statement
ITERATE Statement
REPEAT Statement
WHILE Statement
11.9 Cursors
Declaring Cursors
Cursor OPEN Statement
Cursor FETCH Statement
Cursor CLOSE Statement
11.10 Summary
11.11 Terminal Questions
11.12 Answers
11.1 Introduction
This unit introduces to the user various control statements used within
stored procedures or functions of MySQL server. It starts with a description
of the BeginEnd statements to be written inside a procedure to indicate
the begin and end of a block of statements. It describes the Declare
statement used to declare variables used in a Procedure or Function. It also
discusses the importance of the Variable SET statement. It explains th
eusageof SELECT.INTO statement used to fill in the data from a query
into the current procedure or function. It also discusses about the Declare
Sikkim Manipal University
Unit 11
Stored routines may contain multiple statements, using a BEGIN ... END
compound statement. begin_label and end_label must be the same, if both
are specified. Please note that the optional [NOT] ATOMIC clause is not yet
supported. This means that no transactional savepoint is set at the start of
the instruction block and the BEGIN clause used in this context has no
effect on the current transaction.
Multiple statements requires that a client is able to send query strings
containing ;. This is handled in the mysql command-line client with the
delimiter command. Changing the ; end-of-query delimiter (for example, to
|) allows ; to be used in a routine body.
Self Assessment Questions
1. In a BEGINEND compound statement, the begin_label and end_label
must be the ____, if both are specified.
Unit 11
Unit 11
FOR
condition_value[,...]
This statement specifies handlers that each may deal with one or more
conditions. If one of these conditions occurs, the specified statement is
executed.
For a CONTINUE handler, execution of the current routine continues after
execution of the handler statement. For an EXIT handler, execution of the
Sikkim Manipal University
Unit 11
Example:
mysql> CREATE TABLE test.t (s1 int,primary key (s1));
Query OK, 0 rows affected (0.00 sec)
mysql> delimiter //
mysql> CREATE PROCEDURE handlerdemo ()
-> BEGIN
-> DECLARE CONTINUE HANDLER FOR SQLSTATE '23000' SET @x2 = 1;
-> set @x = 1;
-> INSERT INTO test.t VALUES (1);
-> set @x = 2;
-> INSERT INTO test.t VALUES (1);
-> SET @x = 3;
-> END;
-> //
Query OK, 0 rows affected (0.00 sec)
mysql> CALL handlerdemo()//
Query OK, 0 rows affected (0.00 sec)
mysql> SELECT @x//
+------+
| @x |
+------+
|3|
+------+
1 row in set (0.00 sec)
Notice that @x is 3, which shows that MySQL executed to the end of the
procedure. If the line DECLARE CONTINUE HANDLER FOR SQLSTATE
'23000' SET @x2 = 1; had not been present, MySQL would have taken the
default (EXIT) path after the second INSERT failed due to the PRIMARY
KEY constraint, and SELECT @x would have returned 2.
Unit 11
Unit 11
ITERATE can only appear within LOOP, REPEAT, and WHILE statements.
ITERATE means do the loop iteration again.
Example:
CREATE PROCEDURE doiterate(p1 INT)
BEGIN
label1: LOOP
SET p1 = p1 + 1;
IF p1 < 10 THEN ITERATE label1; END IF;
LEAVE label1;
END LOOP label1;
SET @x = p1;
END
Unit 11
Example:
mysql> delimiter //
mysql> CREATE PROCEDURE dorepeat(p1 INT)
-> BEGIN
-> SET @x = 0;
-> REPEAT SET @x = @x + 1; UNTIL @x > p1 END REPEAT;
-> END
-> //
Query OK, 0 rows affected (0.00 sec)
mysql> CALL dorepeat(1000)//
Query OK, 0 rows affected (0.00 sec)
mysql> SELECT @x//
+------+
| @x |
+------+
| 1001 |
+------+
1 row in set (0.00 sec)
Unit 11
11.9 Cursors
Simple cursors are supported inside stored procedures and functions. The
syntax is as in embedded SQL. Cursors are currently asensitive, read-only,
and non-scrolling. Asensitive means that the server may or may not make a
copy of its result table.
Example:
CREATE PROCEDURE curdemo()
BEGIN
DECLARE done INT DEFAULT 0;
DECLARE CONTINUE HANDLER FOR SQLSTATE '02000' SET done = 1;
DECLARE cur1 CURSOR FOR SELECT id,data FROM test.t1;
DECLARE cur2 CURSOR FOR SELECT i FROM test.t2;
DECLARE a CHAR(16);
DECLARE b,c INT;
OPEN cur1;
OPEN cur2;
REPEAT
FETCH cur1 INTO a, b;
FETCH cur2 INTO c;
IF NOT done THEN
IF b < c THEN
INSERT INTO test.t3 VALUES (a,b);
ELSE
INSERT INTO test.t3 VALUES (a,c);
END IF;
END IF;
UNTIL done END REPEAT;
CLOSE cur1;
CLOSE cur2;
END
Multiple cursors may be defined in a routine, but each must have a unique
name.
11.9.2 Cursor OPEN Statement
OPEN cursor_name
Unit 11
This statement fetches the next row (if a row exists) using the specified
open cursor, and advances the cursor pointer.
11.9.4 Cursor CLOSE Statement
CLOSE cursor_name
11.10 Summary
This unit covers various control statements used within stored procedures or
functions of MySQL server.
1. BEGIN ... END Compound Statement: This topic discusses the
BeginEnd Compound statements used to mark the Begin and End of
a block structures within a stored procedure or function.
2. DECLARE Statement: This statement is used to declare the necessary
variables within a stored procedure or a function
3. Variables in Stored Procedures: This topic covers the declaration of
local variables within a stored procedure or a function whose scope is
within the BeginEnd Statement.
4. Variable SET Statement: This allows an extended syntax of SET a=x,
b=y, ... where different variable types (locally declared variables, server
variables, and global and session server variables) can be mixed.
5. SELECT ... INTO Statement: This statement is also extremely useful
when used in combination with cursors.
6. DECLARE Conditions and Handlers: Certain conditions may require
specific handling. These conditions can relate to errors, as well as
general flow control inside a routine.
7. Flow Control Constructs: The IF, CASE, LOOP, WHILE, ITERATE,
and LEAVE constructs are fully implemented. These constructs may
each contain either a single statement, or a block of statements using
the BEGIN ... END compound statement. Constructs may be nested.
Unit 11
11.12 Answers
Self Assessment Questions
1. same
2. DECLARE
3. SET
4. IF
5. SELECT
6. LEAVE
Terminal Questions
1. (Refer to Section 11.2)
[begin_label:] BEGIN
statement(s)
END [end_label]
Unit 12
Unit 12
Structure
12.1 Introduction
Objectives
12.2 Adding New User Accounts to MySQL
12.3 MySQL Usernames and Passwords
12.4 Securing the Initial MySQL Accounts
12.5 Removing User Accounts from MySQL
12.6 Limiting Account Resources
12.7 Assigning Account Passwords
12.8 Keeping Your Password Secure
12.9 Account Management Statements
DROP USER Syntax
GRANT and REVOKE Syntax
SET PASSWORD Syntax
12.10 Summary
12.11 Terminal Questions
12.12 Answers
12.1 Introduction
This unit describes how to set up accounts for clients of your MySQL server.
It discusses the meaning of account names and passwords as used in
MySQL and how that compares to names and passwords used by your
operating system. It also deals with how to set up new accounts and remove
existing accounts and to change passwords. It contains guidelines for using
passwords securely and how to use secure connections with SSL.
Objectives
After studying this unit, you should be able to:
explain the importance and processes involved in Managing the user
accounts
discuss the methods of adding New user accounts in MySQL
describe the procedure followed in creation of individual user names and
passwords
Unit 12
If you have assigned a password to the root account, you'll also need to
supply a password or -p option for this mysql command and also for those
later in this section.
Unit 12
After connecting to the server as root, you can add new accounts. The
following statements use GRANT to set up four new accounts:
mysql> GRANT ALL PRIVILEGES ON *.* TO 'monty'@'localhost'
-> IDENTIFIED BY 'some_pass' WITH GRANT OPTION;
mysql> GRANT ALL PRIVILEGES ON *.* TO 'monty'@'%'
-> IDENTIFIED BY 'some_pass' WITH GRANT OPTION;
mysql> GRANT RELOAD,PROCESS ON *.* TO 'admin'@'localhost';
mysql> GRANT USAGE ON *.* TO 'dummy'@'localhost';
Unit 12
setting all the global privileges to 'N'. It is assumed that you will grant
specific privileges to the account later.
As an alternative to GRANT, you can create the same accounts directly by
issuing INSERT statements and then telling the server to reload the grant
tables using FLUSH PRIVILEGES:
shell> mysql --user=root mysql
mysql> INSERT INTO user
-> VALUES('localhost','monty',PASSWORD('some_pass'),
-> 'Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y');
mysql> INSERT INTO user
-> VALUES('%','monty',PASSWORD('some_pass'),
-> 'Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y','Y');
mysql> INSERT INTO user SET Host='localhost',User='admin',
-> Reload_priv='Y', Process_priv='Y';
mysql> INSERT INTO user (Host,User,Password)
-> VALUES('localhost','dummy','');
mysql> FLUSH PRIVILEGES;
The reason for using FLUSH PRIVILEGES when you create accounts with
INSERT is to tell the server to re-read the grant tables. Otherwise, the
changes go unnoticed until you restart the server. With GRANT, FLUSH
PRIVILEGES is unnecessary.
The reason for using the PASSWORD() function with INSERT is to encrypt
the password. The GRANT statement encrypts the password for you, so
PASSWORD() is unnecessary.
The 'Y' values enable privileges for the accounts.
Depending on your MySQL version, you may have to use a different number
of 'Y' values in the first two INSERT statements. For the admin account, you
may also employ the more readable extended INSERT syntax using SET.
In the INSERT statement for the dummy account, only the Host, User, and
Password columns in the user table row are assigned values. None of the
privilege columns are set explicitly, so MySQL assigns them all the default
value of 'N'. This is equivalent to what GRANT USAGE does.
Note that to set up a superuser account, it is necessary only to create a user
table entry with the privilege columns set to 'Y'. user table privileges are
global, so no entries in any of the other grant tables are needed.
Unit 12
The next examples create three accounts and give them access to specific
databases. Each of them has a username of custom and password of
obscure.
To create the accounts with GRANT, use the following statements:
shell> mysql --user=root mysql
mysql> GRANT SELECT,INSERT,UPDATE,DELETE,CREATE,DROP
-> ON bankaccount.*
-> TO 'custom'@'localhost'
-> IDENTIFIED BY 'obscure';
mysql> GRANT SELECT,INSERT,UPDATE,DELETE,CREATE,DROP
-> ON expenses.*
-> TO 'custom'@'whitehouse.gov'
-> IDENTIFIED BY 'obscure';
mysql> GRANT SELECT,INSERT,UPDATE,DELETE,CREATE,DROP
-> ON customer.*
-> TO 'custom'@'server.domain'
-> IDENTIFIED BY 'obscure';
Unit 12
The first three INSERT statements add user table entries that allow the user
custom to connect from the various hosts with the given password, but grant
no global privileges (all privileges are set to the default value of 'N'). The
next three INSERT statements add db table entries that grant privileges to
custom for the bankaccount, expenses, and customer databases, but only
when accessed from the proper hosts. As usual when you modify the grant
tables directly, you must tell the server to reload them with FLUSH
PRIVILEGES so that the privilege changes take effect.
If you want to give a specific user access from all machines in a given
domain (for example, mydomain.com), you can issue a GRANT statement
that uses the % wildcard character in the host part of the account name:
mysql> GRANT ...
-> ON *.*
-> TO 'myname'@'%.mydomain.com'
-> IDENTIFIED BY 'mypass';
Unit 12
Unit 12
There must be no space between the -p option and the following password
value.
The preceding commands include the password value on the command line,
which can be a security risk. To avoid this problem, specify the --password
or -p option without any following password value:
shell> mysql --user=monty --password db_name
shell> mysql -u monty -p db_name
When the password option has no password value, the client program prints
a prompt and waits for you to enter the password. (In these examples,
db_name is not interpreted as a password because it is separated from the
preceding password option by a space.)
On some systems, the library routine that MySQL uses to prompt for a
password automatically limits the password to eight characters. That is a
problem with the system library, not with MySQL. Internally, MySQL doesn't
have any limit for the length of the password. To work around the problem,
change your MySQL password to a value that is eight or fewer characters
long, or put your password in an option file.
Self Assessment Questions
3. The ______, as used by MySQL for authentication purposes, have
nothing to do with usernames (login names) as used by Windows or
Unix.
4. MySQL encrypts passwords using ____ algorithm.
Unit 12
The grant tables define the initial MySQL user accounts and their access
privileges. These accounts are set up as follows:
Accounts with the username root are created. These are superuser
accounts that can do anything. The initial root account passwords are
empty, so anyone can connect to the MySQL server as root without a
password and be granted all privileges.
o On Windows, one root account is created; this account allows
connecting from the local host only. The Windows installer will
optionally create an account allowing for connections from any host
only if the user selects the Enable root access from remote
machines option during installation.
o On Unix, both root accounts are for connections from the local host.
Connections must be made from the local host by specifying a
hostname of localhost for one of the accounts, or the actual
hostname or IP number for the other.
Two anonymous-user accounts are created, each with an empty
username. The anonymous accounts have no password, so anyone can
use them to connect to the MySQL server.
o On Windows, one anonymous account is for connections from the
local host. It has all privileges, just like the root accounts. The other
is for connections from any host and has all privileges for the test
database and for other databases with names that start with test.
o On Unix, both anonymous accounts are for connections from the
local host. Connections must be made from the local host by
specifying a hostname of localhost for one of the accounts, or the
actual hostname or IP number for the other. These accounts have all
privileges for the test database and for other databases with names
that start with test_.
As noted, none of the initial accounts have passwords. This means that your
MySQL installation is unprotected until you do something about it:
If you want to prevent clients from connecting as anonymous users
without a password, you should either assign a password to each
anonymous account or else remove the accounts.
You should assign a password to each MySQL root account.
The following instructions describe how to set up passwords for the initial
MySQL accounts, first for the anonymous accounts and then for the root
Sikkim Manipal University
Unit 12
accounts. Replace newpwd in the examples with the actual password that
you want to use. The instructions also cover how to remove the anonymous
accounts, should you prefer not to allow anonymous access at all.
You might want to defer setting the passwords until later, so that you don't
need to specify them while you perform additional setup or testing.
However, be sure to set them before using your installation for production
purposes.
Anonymous Account Password Assignment
To assign passwords to the anonymous accounts, connect to the server as
root and then use either SET PASSWORD or UPDATE. In either case, be
sure to encrypt the password using the PASSWORD() function.
To use SET PASSWORD on Windows, do this:
shell> mysql -u root
mysql> SET PASSWORD FOR ''@'localhost' = PASSWORD('newpwd');
mysql> SET PASSWORD FOR ''@'%' = PASSWORD('newpwd');
Look for the record that has root in the User column and something other
than localhost in the Host column. Then use that Host value in the second
SET PASSWORD statement.
The other way to assign passwords to the anonymous accounts is by using
UPDATE to modify the user table directly. Connect to the server as root and
issue an UPDATE statement that assigns a value to the Password column
of the appropriate user table records. The procedure is the same for
Sikkim Manipal University
Unit 12
After you update the passwords in the user table directly using UPDATE,
you must tell the server to re-read the grant tables with FLUSH
PRIVILEGES. Otherwise, the change goes unnoticed until you restart the
server.
Anonymous Account Removal
If you prefer to remove the anonymous accounts instead, do so as follows:
shell> mysql -u root
mysql> DELETE FROM mysql.user WHERE User = '';
mysql> FLUSH PRIVILEGES;
That account allows anonymous access but has full privileges, so removing
it improves security.
Root Account Password Assignment
You can assign passwords to the root accounts in several ways. The
following discussion demonstrates three methods:
1. Use the SET PASSWORD statement
2. Use the mysqladmin command-line client program
3. Use the UPDATE statement
To assign passwords using SET PASSWORD, connect to the server as root
and issue two SET PASSWORD statements. Be sure to encrypt the
password using the PASSWORD() function.
Unit 12
Unit 12
Unit 12
completely (including all of its privileges), you should use the following
procedure, performing these steps in the order shown:
1. Use SHOW GRANTS to determine what privileges the account has.
2. Use REVOKE to revoke the privileges displayed by SHOW GRANTS.
This removes rows for the account from all the grant tables except the
user table, and revokes any global privileges listed in the user table.
3. Delete the account by using DROP USER to remove the user table
record.
Note: DROP USER does not automatically close any open user sessions.
Rather, in the event that a user with an open session is dropped, the
statement does not take effect until that user's session is closed. Once the
session is closed, the user is dropped, and that user's next attempt to log in
will fail. This is by design.
Self Assessment Questions
7. To use it, you must have the global _____ privilege or the _____
privilege for the mysql database.
Unit 12
An account in this context is a single row in the user table. Each account is
uniquely identified by its User and Host column values.
As a prerequisite for using this feature, the user table in the mysql database
must contain the resource-related columns. Resource limits are stored in the
max_questions, max_updates, max_connections, and max_user_
connections columns. If your user table doesn't have these columns, it must
be upgraded.
To set resource limits with a GRANT statement, use a WITH clause that
names each resource to be limited and a per-hour count indicating the limit
value. For example, to create a new account that can access the customer
database, but only in a limited fashion, issue this statement:
mysql> GRANT ALL ON customer.* TO 'francis'@'localhost'
-> IDENTIFIED BY 'frank'
-> WITH MAX_QUERIES_PER_HOUR 20
->
MAX_UPDATES_PER_HOUR 10
->
MAX_CONNECTIONS_PER_HOUR 5
->
MAX_USER_CONNECTIONS 2;
The limit types need not all be named in the WITH clause, but those named
can be present in any order. The value for each per-hour limit should be an
integer representing a count per hour. If the GRANT statement has no WITH
clause, the limits are each set to the default value of zero (that is, no limit).
For MAX_USER_CONNECTIONS, the limit is an integer indicating the
maximum number of simultaneous connections the account can make at
any one time. If the limit is set to the default value of zero, the
max_user_connections system variable determines the number of
simultaneous connections for the account.
To set or change limits for an existing account, use a GRANT USAGE
statement at the global level (ON *.*). The following statement changes the
query limit for francis to 100:
mysql> GRANT USAGE ON *.* TO 'francis'@'localhost'
-> WITH MAX_QUERIES_PER_HOUR 100;
Unit 12
To remove an existing limit, set its value to zero. For example, to remove
the limit on how many times per hour francis can connect, use this
statement:
mysql> GRANT USAGE ON *.* TO 'francis'@'localhost'
-> WITH MAX_CONNECTIONS_PER_HOUR 0;
Resource-use counting takes place when any account has a non-zero limit
placed on its use of any of the resources.
As the server runs, it counts the number of times each account uses
resources. If an account reaches its limit on number of connections within
the last hour, further connections for the account are rejected until that hour
is up. Similarly, if the account reaches its limit on the number of queries or
updates, further queries or updates are rejected until the hour is up. In all
such cases, an appropriate error message is issued.
Resource counting is done per account, not per client. For example, if your
account has a query limit of 50, you cannot increase your limit to 100 by
making two simultaneous client connections to the server. Queries issued
on both connections are counted together.
The current per-hour resource-use counts can be reset globally for all
accounts, or individually for a given account:
To reset the current counts to zero for all accounts, issue a FLUSH
USER_RESOURCES statement. The counts also can be reset by
reloading the grant tables (for example, with a FLUSH PRIVILEGES
statement or a mysqladmin reload command).
The counts for an individual account can be set to zero by re-granting it
any of its limits. To do this, use GRANT USAGE as described earlier and
specify a limit value equal to the value that the account currently has.
Counter resets do not affect the MAX_USER_CONNECTIONS limit.
All counts begin at zero when the server starts; counts are not carried over
through a restart.
Self Assessment Questions
8. The setting ___________ method is strictly global, and does not allow
for management of individual accounts.
Unit 12
The account for which this command resets the password is the one with a
user table row that matches user_name in the User column and the client
host from which you connect in the Host column.
Another way to assign a password to an account is to issue a SET
PASSWORD statement:
mysql> SET PASSWORD FOR 'jeffrey'@'%' = PASSWORD('biscuit');
Only users such as root that have update access to the mysql database can
change the password for other users. If you are not connected as an
anonymous user, you can change your own password by omitting the FOR
clause:
mysql> SET PASSWORD = PASSWORD('biscuit');
You can also use a GRANT USAGE statement at the global level (ON *.*) to
assign a password to an account without affecting the account's current
privileges:
mysql> GRANT USAGE ON *.* TO 'jeffrey'@'%' IDENTIFIED BY 'biscuit';
Unit 12
To change the password for an existing account, use UPDATE to set the
Password column value:
shell> mysql -u root mysql
mysql> UPDATE user SET Password = PASSWORD('bagel')
-> WHERE Host = '%' AND User = 'francis';
mysql> FLUSH PRIVILEGES;
The result is that the literal value 'biscuit' is stored as the password in the
user table, not the encrypted value. When jeffrey attempts to connect to the
server using this password, the value is encrypted and compared to the
value stored in the user table. However, the stored value is the literal string
'biscuit', so the comparison fails and the server rejects the connection:
shell> mysql -u jeffrey -pbiscuit test
Access denied
Unit 12
you run client programs are listed here, along with an assessment of the
risks of each method:
Use a -pyour_pass or password=your_pass option on the command
line. For example:
shell> mysql -u francis -pfrank db_name
Unit 12
If you store your password in .my.cnf, the file should not be accessible to
anyone but yourself. To ensure this, set the file access mode to 400 or
600. For example:
shell> chmod 600 .my.cnf
The DROP USER statement deletes a MySQL account that doesnt have
any privileges. It serves to remove the account record from the user table.
The account is named using the same format as for GRANT or REVOKE;
for example, 'jeffrey'@'localhost'. The user and host parts of the account
name correspond to the User and Host column values of the user table
record for the account.
To remove a MySQL user account, you should use the following procedure,
performing the steps in the order shown:
1. Use SHOW GRANTS to determine what privileges the account has.
Unit 12
GRANT
OPTION
FROM
user_name
Unit 12
Table level
Table privileges apply to all columns in a given table. These privileges are
stored in the mysql.tables_priv table. GRANT ALL ON db.table and
REVOKE ALL ON db.table grant and revoke only table privileges.
Column level
Column privileges apply to single columns in a given table. These privileges
are stored in the mysql.columns_priv table. When using REVOKE you must
specify the same columns that were granted.
To make it easy to revoke all privileges for a user, MySQL 4.1.2 has added
the syntax:
REVOKE ALL PRIVILEGES,
[, user_name ...]
GRANT
OPTION
FROM
user_name
This drops all database, table, and column level privileges for the user.
For the GRANT and REVOKE statements, priv_type can be specified as
any of the following:
Table 12.1 Privilege Types and Descriptions
Privilege Type
Description
ALL [PRIVILEGES]
ALTER
CREATE
DELETE
DROP
EXECUTE
FILE
INDEX
INSERT
LOCK TABLES
PROCESS
Unit 12
REFERENCES
RELOAD
REPLICATION CLIENT
REPLICATION SLAVE
SELECT
SHOW DATABASES
SHUTDOWN
SUPER
UPDATE
USAGE
GRANT OPTION
USAGE can be used when you want to create a user that has no privileges.
The privileges CREATE TEMPORARY TABLES, EXECUTE, LOCK
TABLES, REPLICATION ..., SHOW DATABASES and SUPER are new for
in MySQL 4.0.2. To use these new privileges after upgrading to 4.0.2, you
must run the mysql_fix_privilege_tables script.
In older MySQL versions that do not have the SUPER privilege, the
PROCESS privilege can be used instead.
To revoke the GRANT OPTION privilege from a user, use a priv_type value
of GRANT OPTION:
mysql> REVOKE GRANT OPTION ON ... FROM ...;
The only priv_type values you can specify for a table are SELECT, INSERT,
UPDATE, DELETE, CREATE, DROP, GRANT OPTION, INDEX, and
ALTER.
The only priv_type values you can specify for a column (that is, when you
use a column_list clause) are SELECT, INSERT, and UPDATE.
Sikkim Manipal University
Unit 12
MySQL allows you to create database level privileges even if the database
doesnt exist, to make it easy to prepare for database use. However, MySQL
currently does not allow you to create table level grants if the table doesnt
exist.
MySQL will not automatically revoke any privileges even if you drop a table
or drop a database.
You can set global privileges by using ON *.* syntax. You can set database
privileges by using ON db_name.* syntax. If you specify ON * and you have
a current database, the privileges will be granted in that database. (Warning:
If you specify ON * and you dont have a current Database, the privileges
granted will be global!)
12.9.3 SET PASSWORD Syntax
SET PASSWORD = PASSWORD('some password')
SET PASSWORD FOR user = PASSWORD('some password')
PASSWORD('newpass');
Unit 12
12.10 Summary
This unit covers the topics of setting user accounts and privileges for
individual accounts for clients of your MySQL server.
1. Adding New User Accounts to MySQL: You can create MySQL
accounts in two ways:
By using statements intended for creating accounts, such as
CREATE USER or GRANT
By manipulating the MySQL grant tables directly with statements
such as INSERT, UPDATE, or DELETE
The preferred method is to use account-creation statements because they
are more concise and less error-prone.
2. MySQL Usernames and Passwords: A MySQL account is defined in
terms of a username and the client host or hosts from which the user
can connect to the server. The account also has a password. There are
several distinctions between the way usernames and passwords are
used by MySQL and the way they are used by your operating system
3. Securing the Initial MySQL Accounts: The grant tables define the
initial MySQL user accounts and their access privileges. The stting up of
the accounts and corresponding passwords is discussed here.
4. Removing User Accounts from MySQL: The DROP USER statement
removes one or more MySQL accounts. To use it, you must have the
global CREATE USER privilege or the DELETE privilege for the mysql
database.
5. Limiting Account Resources: One means of limiting use of MySQL
server resources is to set the max_user_connections system variable to
a non-zero value. However, this method is strictly global, and does not
allow for management of individual accounts. In addition, it limits only
the number of simultaneous connections made using a single account,
and not what a client can do once connected. Both types of control are
interest to many MySQL administrators, particularly those working for
Internet Service Providers.
6. Assigning Account Passwords: Passwords may be assigned from the
command line by using the mysqladmin command. The syntax and
corresponding settings in this regard are discussed here.
7. Keeping Your Password Secure: When you run a client program to
connect to the MySQL server, it is inadvisable to specify your password
Sikkim Manipal University
Unit 12
12.12 Answers
Self Assessment Questions
1. FLUSH
2. mysql>mysql user = root mysql
3. Usernames
4. its own
5. pre-initialized
6. Anonymous
7. CREATE USER or DELETE
8. setting of max_user_connections variable to a zero value
9. mysqladmin
Terminal Questions
1. You can create MySQL accounts in two ways:
By using statements intended for creating accounts, such as
CREATE USER or GRANT
By manipulating the MySQL grant tables directly with statements
such as INSERT, UPDATE, or DELETE
The preferred method is to use account-creation statements because
they are more concise and less error-prone.
Unit 12
Unit 12
Unit 13
Unit 13
Structure
13.1 Introduction
Objectives
13.2 General Security Guidelines
13.3 Making MySQL Secure Against Attackers
13.4 Startup Options for mysqld concerning Security
13.5 Security Issues with LOAD DATA LOCAL
13.6 Summary
13.7 Terminal Questions
13.8 Answers
13.1 Introduction
This unit speaks about the general security guidelines to be followed when
dealing with databases. It discusses the issues of possible attacks on the
databases and the strategies to be followed against the attackers. It
discusses about the mysqld option used during server startup for
maintaining the security. It also describes the security steps to be followed
when loading data from external sources into the database.
Objectives
After studying this unit, you should be able to:
discuss the importance and applications of general security guidelines
explain various procedures followed for making MySQL secure
describe the startup options for mysqld concerning security
discuss various aspects of security by using LOAD DATA LOCAL
Unit 13
host (not just the MySQL server) against all types of applicable attacks:
eavesdropping, altering, playback, and denial of service.
We do not cover all aspects of availability and fault tolerance here.
MySQL uses security based on Access Control Lists (ACLs) for all
connections, queries, and other operations that users can attempt to
perform. There is also support for SSL-encrypted connections between
MySQL clients and servers. Many of the concepts discussed here are not
specific to MySQL at all; the same general ideas apply to almost all
applications.
When running MySQL, follow these guidelines whenever possible:
Do not ever give anyone (except MySQL root accounts) access to
the user table in the mysql database! This is critical.
Unit 13
and type, but difficult to guess for someone who does not know the
sentence.
Invest in a firewall. This protects you from at least 50% of all types of
exploits in any software. Put MySQL behind the firewall or in a
demilitarized zone (DMZ).
Checklist:
Try to scan your ports from the Internet using a tool such as nmap.
MySQL uses port 3306 by default. This port should not be accessible
from untrusted hosts. Another simple way to check whether or not
your MySQL port is open is to try the following command from some
remote machine, where server_host is the hostname or IP number of
the host on which your MySQL server runs:
shell> telnet server_host 3306
If you get a connection and some garbage characters, the port is
open, and should be closed on your firewall or router, unless you
really have a good reason to keep it open. If telnet hangs or the
connection is refused, the port is blocked, which is how you want it to
be.
Do not trust any data entered by users of your applications. They can try
to trick your code by entering special or escaped character sequences in
Web forms, URLs, or whatever application you have built. Be sure that
your application remains secure if a user enters something like ; DROP
DATABASE mysql;. This is an extreme example, but large security
leaks and data loss might occur as a result of hackers using similar
techniques, if you do not prepare for them.
A common mistake is to protect only string data values. Remember to
check numeric data as well. If an application generates a query such as
SELECT * FROM table WHERE ID=234 when a user enters the value
234, the user can enter the value 234 OR 1=1 to cause the application
to generate the query SELECT * FROM table WHERE ID=234 OR 1=1.
As a result, the server retrieves every row in the table. This exposes
every row and causes excessive server load. The simplest way to
protect from this type of attack is to use single quotes around the
numeric constants: SELECT * FROM table WHERE ID='234'. If the user
enters extra information, it all becomes part of the string. In a numeric
Sikkim Manipal University
Unit 13
Unit 13
Unit 13
This causes the server to start as the designated user whether you start
it manually or by using mysqld_safe or mysql.server.
Unit 13
Unit 13
--allow-suspicious-udfs
This option controls whether user-defined functions that have only an
xxx symbol for the main function can be loaded. By default, the option is
off and only UDFs that have at least one auxiliary symbol can be loaded;
this prevents attempts at loading functions from shared object files other
than those containing legitimate UDFs.
--local-infile[={0|1}]
If you start the server with --local-infile=0, clients cannot use LOCAL in
LOAD DATA statements.
--old-passwords
Force the server to generate short (pre-4.1) password hashes for new
passwords. This is useful for compatibility when the server must support
older client programs.
--safe-show-database (OBSOLETE)
In previous versions of MySQL, this option caused the SHOW
DATABASES statement to display the names of only those databases
Unit 13
for which the user had some kind of privilege. In MySQL 5.1, this option
is no longer available as this is now the default behavior, and there is a
SHOW DATABASES privilege that can be used to control access to
database names on a per-account basis.
--safe-user-create
If this option is enabled, a user cannot create new MySQL users by
using the GRANT statement unless the user has the INSERT privilege
for the mysql.user table or any column in the table. If you want a user to
have the ability to create new users that have those privileges that the
user has the right to grant, you should grant the user the following
privilege:
GRANT INSERT(user) ON mysql.user TO 'user_name'@'host_name';
This ensures that the user cannot change any privilege columns directly,
but has to use the GRANT statement to give privileges to other users.
--secure-auth
Disallow authentication for accounts that have old (pre-4.1) passwords.
The mysql client also has a --secure-auth option, which prevents
connections to a server if the server requires a password in old format
for the client account.
--secure-file-priv=path
This option limits the effect of the LOAD_FILE() function and the LOAD
DATA and SELECT ... INTO OUTFILE statements to work only with files
in the specified directory. This option was added in MySQL 5.1.17.
--skip-grant-tables
This option causes the server not to use the privilege system at all. This
gives anyone with access to the server unrestricted access to all
databases. You can cause a running server to start using the grant
tables again by executing mysqladmin flushprivileges or mysqladmin
reload command from a system shell, or by issuing a MySQL FLUSH
PRIVILEGES statement. This option also suppresses loading of plugins
and user-defined functions (UDFs).
--skip-merge
Disable the MERGE storage engine. This option was added in MySQL
5.1.12. It can be used if the following behavior is undesirable:
Unit 13
If a user has access to MyISAM table t, that user can create a MERGE
table m that accesses t. However, if the user's privileges on t are
subsequently revoked, the user can continue to access t by doing so
through m.
--skip-name-resolve
Hostnames are not resolved. All Host column values in the grant tables
must be IP numbers or localhost.
--skip-networking
Do not allow TCP/IP connections over the network. All connections to
mysqld must be made via Unix socket files.
--skip-show-database
With this option, the SHOW DATABASES statement is allowed only to
users who have the SHOW DATABASES privilege, and the statement
displays all database names. Without this option, SHOW DATABASES
is allowed to all users, but displays each database name only if the user
has the SHOW DATABASES privilege or some privilege for the
database. Note that any global privilege is a privilege for the database.
--ssl*
Options that begin with --ssl specify whether to allow clients to connect
via SSL and indicate where to find SSL keys and certificates.
Unit 13
Such a server could access any file on the client host to which the client
user has read access.
If you build MySQL from source but do not invoke configure with the -enable-local-infile option, LOAD DATA LOCAL cannot be used by any
client
unless
it
is
written
explicitly
to
invoke
mysql_options(...MYSQL_OPT_LOCAL_INFILE, 0).
You can disable all LOAD DATA LOCAL commands from the server side
by starting mysqld with the --local-infile=0 option.
For the mysql command-line client, LOAD DATA LOCAL can be enabled
by specifying the --local-infile[=1] option, or disabled with the --localinfile=0 option. Similarly, for mysqlimport, the --local or -L option enables
local data file loading. In any case, successful use of a local loading
operation requires that the server is enabled to allow it.
If you use LOAD DATA LOCAL in Perl scripts or other programs that
read the [client] group from option files, you can add the local-infile=1
option to that group. However, to keep this from causing problems for
programs that do not understand local-infile, specify it using the looseprefix:
[client]
loose-local-infile=1
Unit 13
13.6 Summary
This unit speaks about the general security guidelines to be followed when
dealing with databases. It discusses the issues of possible attacks on the
databases and the strategies to be followed against the attackers.
1. General Security Guidelines: This section describes some general
security issues to be aware of and what you can do to make your
MySQL installation more secure against attack or misuse.
2. Making MySQL Secure Against Attackers: When you connect to a
MySQL server, you should use a password. The password is not
transmitted in clear text over the connection. The methods of
transmitting the text based passwords in encrypted format is discussed
here.
3. Startup Options for mysqld concerning Security: It describes various
security options and variables available with mysqld for secure
transmission of data.
4. Security Issues with LOAD DATA LOCAL: It covers the theory behind
the Security issues and their handling with the Load Data Local Option.
13.8 Answers
Self Assessment Questions
1. Access Control Lists (ACLs)
2. 3306
3. compressed
4. FILE
5. --safe-user-create
Terminal Questions
1. The following are the general security guidelines to be followed:
a. Do not ever give anyone (except MySQL root accounts) access to
the user table in the mysql database!
b. Learn the MySQL access privilege system.
c. Do not store any plain-text passwords in your database.
Sikkim Manipal University
Unit 13
Unit 14
Unit 14
Log Files
Structure
14.1 Introduction
Objectives
14.2 Error Log
14.3 The General Query Log
14.4 The Binary Log
14.5 The Slow Query Log
14.6 Log File Maintenance
14.7 Summary
14.8 Terminal Questions
14.9 Answers
14.1 Introduction
This unit deals with administration of historic information using log files
regarding the transaction done with the database server. .MySQL has
several different logs that can help you find out what is going on inside
mysqld:
Table 14.1: Log Types
Log Type
By default, all log files are created in the mysqld data directory. You can
force mysqld to close and reopen the log files (or in some cases switch to a
new log) by flushing the logs. Log flushing occurs when you issue a FLUSH
LOGS statement or execute mysqladmin flush-logs or mysqladmin refresh.
Unit 14
Objectives
After studying this unit, you should be able to:
Define and discuss the importance of using Log files in MySQL
Describe the importance and usage of Error logs
Describe the importance and usage of General Query and Binary logs
Explain the usage of Slow Query log
Describe the purpose and usage of Log File Maintenance
Unit 14
prior to 5.1.20. To explicitly specify use of an error log file, specify --logerror=file_name to mysqld_safe, and mysqld_safe will arrange for mysqld to
write messages to a log file. To use syslog instead, specify the --syslog
option.
Self Assessment Questions
1. The _____ log is used to establish client connections and statements
received from clients.
2. The _____ option specifies the destination for log output, if logging is
enabled, but the option does not in itself enable the logs.
3. If you execute ____ LOGS, the error log is renamed with the suffix -old
and mysqld creates a new empty log file.
Unit 14
If you specify no filename for the general query log, the default name is
host_name.log in the data directory. If you specify a filename that is not an
absolute pathname, the server writes the file in the data directory.
When --log or -l is specified, the --general_log option also may be given as
of MySQL 5.1.12 to specify the initial general query log state. With no
argument or an argument of 0, the option disables the log. If omitted or
given with an argument of 1, the option enables the log.
For runtime control of the general query log, use the global general_log and
general_log_file system variables. Set general_log to 0 (or OFF) to disable
the log or to 1 (or ON) to enable it. Set general_log_file to specify the name
of the log file. If a log file already is open, it is closed and the new file is
opened.
When the general query log is enabled, output is written to any destinations
specified by the --log-output option or log_output system variable. If you
enable the log, the server opens the log file and writes startup messages to
it. However, logging of queries to the file does not occur unless the FILE log
destination is selected. If the destination is NONE, no queries are written
even if the general log is enabled. Setting the log filename has no effect on
logging if the log destination value does not contain FILE.
Server restarts and log flushing do not cause a new general query log file to
be generated (although flushing closes and reopens it). On Unix, you can
rename the file and create a new one by using the following commands:
shell> mv host_name.log host_name-old.log
shell> mysqladmin flush-logs
shell> cp host_name-old.log backup-directory
shell> rm host_name-old.log
Before 5.1.3, you cannot rename a log file on Windows while the server has
it open. You must stop the server and rename the file, and then restart the
server to create a new log file. As of 5.1.3, this applies only to the error log.
However, a stop and restart can be avoided by using FLUSH LOGS, which
causes the server to rename the error log with an -old suffix and open a new
error log.
Unit 14
As of MySQL 5.1.12, you can disable the general query log at runtime:
SET GLOBAL general_log = 'OFF';
With the log disabled, rename the log file externally; for example, from the
command line. Then enable the log again:
SET GLOBAL general_log = 'ON;
This method works on any platform and does not require a server restart.
The session sql_log_off variable can be set to ON or OFF to disable or
enable general query logging for the current connection.
Unit 14
The format of the events recorded in the binary log is dependent on the
binary logging format. Three format types are supported, rowbased logging,
statement-based logging and mixed-base logging. The binary logging format
used depends on the MySQL version.
Running the server with the binary log enabled makes performance about
1% slower. However, the benefits of the binary log for restore operations
and in allowing you to set up replication generally outweigh this minor
performance decrement.
When started with the --log-bin[=base_name] option, mysqld writes a log file
containing all SQL statements that update data (both DDL and DML
statements). If no base_name value is given, the default name is the value
of the pid-file option (which by default is the name of host machine) followed
by -bin. If the basename is given, but not as an absolute pathname, the
server writes the file in the data directory.
If you supply an extension in the log name (for example, --logbin=base_name.extension), the extension is silently removed and ignored.
mysqld appends a numeric extension to the binary log basename to
generate binary log filenames. The number increases each time the server
creates a new log file, thus creating an ordered series of files. The server
creates a new file in the series each time it starts or flushes the logs. The
server also creates a new binary log file automatically when the current log's
size reaches max_binlog_size.
A binary log file may become larger than max_binlog_size if you are using
large transactions because a transaction is written to the file in one piece,
never split between files.
To keep track of which binary log files have been used, mysqld also creates
a binary log index file that contains the names of all used binary log files. By
default, this has the same basename as the binary log file, with the
extension '.index'. You can change the name of the binary log index file with
the --log-bin-index[=file_name] option. You should not manually edit this file
while mysqld is running; doing so would confuse mysqld.
You can delete all binary log files with the RESET MASTER statement, or a
subset of them with PURGE BINARY LOGS.
Unit 14
Writes to the binary log file and binary log index file are handled in the same
way as writes to MyISAM tables.
The binary log format has some known limitations that can affect recovery
from backups.
A replication slave server by default does not write to its own binary log any
data modifications that are received from the replication master.
To log these modifications, start the slave with the --log-slave-updates
option.
Evaluation of update selection options: The server evaluates the options
for logging or ignoring updates to the binary log according to the following
rules:
1. Are there --binlog-do-db or --binlog-ignore-db rules?
No: Write the statement to the binary log and exit.
Yes: Go to the next step.
2. There are some rules (--binlog-do-db, --binlog-ignore-db, or both). Is
there a default database (has any database been selected by USE?)?
No: Do not write the statement, and exit.
Yes: Go to the next step.
3. There is a default database. Are there some --binlog-do-db rules?
Yes: Does the default database match any of the --binlog-do-db rules?
Yes: Write the statement and exit.
No: Do not write the statement, and exit.
No: Go to the next step.
4. There are some --binlog-ignore-db rules. Does the default database
match any of the --binlog-ignore-db rules?
Yes: Do not write the statement, and exit.
No: Write the query and exit.
Important Note: An exception is made in the rules just given for the
CREATE DATABASE, ALTER DATABASE, and DROP DATABASE
statements. In those cases, the database being created, altered, or dropped
replaces the default database when determining whether to log or ignore
updates.
For example, a slave running with only --binlog-do-db=sales does not write
to the binary log any statement for which the default database is different
Sikkim Manipal University
Unit 14
from sales (in other words, --binlog-do-db can sometimes mean ignore
other databases).
If you are using replication, you should not delete old binary log files until
you are sure that no slave still needs to use them. For example, if your
slaves never run more than three days behind, once a day you can execute
mysqladmin flush-logs on the master and then remove any logs that are
more than three days old. You can remove the files manually, but it is
preferable to use PURGE BINARY LOGS, which also safely updates the
binary log index file for you (and which can take a date argument).
If you are using replication, you should not delete old binary log files until
you are sure that no slave still needs to use them. For example, if your
slaves never run more than three days behind, once a day you can execute
mysqladmin flush-logs on the master and then remove any logs that are
more than three days old. You can remove the files manually, but it is
preferable to use PURGE BINARY LOGS, which also safely updates the
binary log index file for you (and which can take a date argument).
A client that has the SUPER privilege can disable binary logging of its own
statements by using a SET sql_log_bin=0 statement.
You can display the contents of binary log files with the mysqlbinlog utility.
This can be useful when you want to reprocess statements in the log. For
example, you can update a MySQL server from the binary log as follows:
shell> mysqlbinlog log_file | mysql -h server_name
Unit 14
Beginning with MySQL 5.1.23, both of these statements are written to the
binary log. (Bug#23333)
Within an uncommitted transaction, all updates (UPDATE, DELETE, or
INSERT) that change transactional tables such as BDB or InnoDB tables
are cached until a COMMIT statement is received by the server. At that
point, mysqld writes the entire transaction to the binary log before the
COMMIT is executed. When the thread that handles the transaction starts, it
allocates a buffer of binlog_cache_size to buffer statements.
If a statement is bigger than this, the thread opens a temporary file to store
the transaction. The temporary file is deleted when the thread ends.
Modifications to non-transactional tables cannot be rolled back. If a
transaction that is rolled back includes modifications to non-transactional
tables, the entire transaction is logged with a ROLLBACK statement at the
end to ensure that the modifications to those tables are replicated.
The Binlog_cache_use status variable shows the number of transactions
that used this buffer (and possibly a temporary file) for storing statements.
The Binlog_cache_disk_use status variable shows how many of those
transactions actually had to use a temporary file. These two variables can
be used for tuning binlog_cache_size to a large enough value that avoids
the use of temporary files.
The max_binlog_cache_size system variable (default 4GB, which is also the
maximum) can be used to restrict the total size used to cache a multiplestatement transaction. If a transaction is larger than this many bytes, it fails
and rolls back. The minimum value is 4096.
If you are using the binary log and row based logging, concurrent inserts are
converted to normal inserts for CREATE ... SELECT or INSERT ... SELECT
statement. This is done to ensure that you can re-create an exact copy of
your tables by applying the log during a backup operation. If you are using
statement based logging then the original statement is written to the log.
By default, the binary log is not synchronized to disk at each write. So if the
operating system or machine (not only the MySQL server) crashes, there is
a chance that the last statements of the binary log are lost. To prevent this,
Unit 14
you can make the binary log be synchronized to disk after every N writes to
the binary log, with the sync_binlog system variable.
1 is the safest value for sync_binlog, but also the slowest. Even with
sync_binlog set to 1, there is still the chance of an inconsistency between
the table content and binary log content in case of a crash. For example, if
you are using InnoDB tables and the MySQL server processes a COMMIT
statement, it writes the whole transaction to the binary log and then commits
this transaction into InnoDB.
If the server crashes between those two operations, the transaction is rolled
back by InnoDB at restart but still exists in the binary log.
To resolve this, you should set --innodb_support_xa to 1. Although this
option is related to the support of XA transactions in InnoDB, it also ensures
that the binary log and InnoDB data files are synchronized.
For this option to provide a greater degree of safety, the MySQL server
should also be configured to synchronize the binary log and the InnoDB logs
to disk at every transaction. The InnoDB logs are synchronized by default,
and sync_binlog=1 can be used to synchronize the binary log. The effect of
this option is that at restart after a crash, after doing a rollback of
transactions, the MySQL server cuts rolled back InnoDB transactions from
the binary log. This ensures that the binary log reflects the exact data of
InnoDB tables, and so, that the slave remains in synchrony with the master
(not receiving a statement which has been rolled back).
If the MySQL server discovers at crash recovery that the binary log is
shorter than it should have been, it lacks at least one successfully
committed InnoDB transaction. This should not happen if sync_binlog=1 and
the disk/filesystem do an actual sync when they are requested to (some
don't), so the server prints an error message The binary log <name> is
shorter than its expected size. In this case, this binary log is not correct and
replication should be restarted from a fresh snapshot of the master's data.
For MySQL 5.1.20 and later (and MySQL 5.0.46 and later for backward
compatibility), the session values of the following system variables are
written to the binary log and honored by the replication slave when parsing
the binary log:
sql_mode
Sikkim Manipal University
Unit 14
foreign_key_checks
unique_checks
character_set_client
collation_connection
collation_database
collation_server
sql_auto_is_null
Unit 14
If you specify no filename for the slow query log, the default name is
host_name-slow.log in the data directory. If you specify a filename that is
not an absolute pathname, the server writes the file in the data directory.
When --log-slow-queries is specified, --slow_query_log also may be given
as of MySQL 5.1.12 to specify the initial slow query log state. With no
argument or an argument of 0, the option disables the log. If omitted or
given with an argument of 1, the option enables the log.
For runtime control of the general query log, use the global slow_query_log
and slow_query_log_file system variables. Set slow_query_log to 0 (or OFF)
to disable the log or to 1 (or ON) to enable it. Set general_log_file to specify
the name of the log file. If a log file already is open, it is closed and the new
file is opened.
When the slow query log is enabled, output is written to any destinations
specified by the --log-output option or log_output system variable. If you
enable the log, the server opens the log file and writes startup messages to
it. However, logging of queries to the file does not occur unless the FILE log
destination is selected. If the destination is NONE, no queries are written
even if the slow query log is enabled. Setting the log filename has no effect
on logging if the log destination value does not contain FILE.
The slow query log can be used to find queries that take a long time to
execute and are therefore candidates for optimization. However, examining
a long slow query log can become a difficult task. To make this easier, you
can process the slow query log using the mysqldumpslow command to
summarize the queries that appear in the log. Use mysqldumpslow --help to
see the options that this command supports.
In MySQL 5.1, queries that do not use indexes are logged in the slow query
log if the --log-queries-not-using-indexes option is specified.
In MySQL 5.1, the --log-slow-admin-statements server option enables you to
request logging of slow administrative statements such as OPTIMIZE
TABLE, ANALYZE TABLE, and ALTER TABLE to the slow query log.
Queries handled by the query cache are not added to the slow query log,
nor are queries that would not benefit from the presence of an index
because the table has zero rows or one row.
Unit 14
Replication slaves do not write replicated queries to the slow query log,
even if the same queries were written to the slow query log on the master.
This is a known issue which we intend to fix in a future version of MySQL.
(Bug#23300).
Self Assessment Questions
6. The minimum and default values of long_query_time are _____ and
____, respectively.
Unit 14
If binary logging is enabled, the server closes the current binary log file
and opens a new log file with the next sequence number.
If the server was given an error log filename with the --log-error option, it
renames the error log with the suffix -old and creates a new empty error
log file.
The server creates a new binary log file when you flush the logs. However, it
just closes and reopens the general and slow query log files. To cause new
files to be created on Unix, rename the current logs before flushing them. At
flush time, the server will open new logs with the original names. For
example, if the general and slow query logs are named mysql.log and
mysql-slow.log, you can use a series of commands like this:
shell> cd mysql-data-directory
shell> mv mysql.log mysql.old
shell> mv mysql-slow.log mysql-slow.old
shell> mysqladmin flush-logs
At this point, you can make a backup of mysql.old and mysql-slow.log and
then remove them from disk.
Before 5.1.3, you cannot rename a log file on Windows while the server has
it open. You must stop the server and rename the file, and then restart the
server to create a new log file. As of 5.1.3, this applies only to the error log.
However, a stop and restart can be avoided by using FLUSH LOGS, which
causes the server to rename the error log with an -old suffix and open a new
error log.
As of MySQL 5.1.2, you can disable the general query log or slow query log
at runtime:
SET GLOBAL general_log = 'OFF';
SET GLOBAL slow_query_log = 'OFF';
With the logs disabled, rename the log files externally; for example, from the
command line. Then enable the logs again:
SET GLOBAL general_log = 'ON';
SET GLOBAL slow_query_log = 'ON';
This method works on any platform and does not require a server restart.
Sikkim Manipal University
Unit 14
14.7 Summary
MySQL has several different logs that can help you find out what is going on
inside mysqld. By default, all log files are created in the mysqld data
directory. This unit discusses all the log files present in the mysqld option.
1. Error Log: The error log contains information indicating when mysqld
was started and stopped and also any critical errors that occur while the
server is running. If mysqld notices a table that needs to be
automatically checked or repaired, it writes a message to the error log.
2. The General Query Log: The general query log is a general record of
what mysqld is doing. The server writes information to this log when
clients connect or disconnect, and it logs each SQL statement received
from clients. The general query log can be very useful when you suspect
an error in a client and want to know exactly what the client sent to
mysqld.
3. The Binary Log: The binary log contains all statements that update
data. It also contains statements that potentially could have updated it
(for example, a DELETE which matched no rows), unless row-based
logging is used. Statements are stored in the form of events that
describe the modifications. The binary log also contains information
about how long each statement took that updated data.
4. The Slow Query Log: The slow query log consists of all SQL
statements that took more than long_query_time seconds to execute
and (as of MySQL5.1.21) required at least min_examined_row_limit
rows to be examined. The time to acquire the initial table locks is not
counted as execution time.
5. Log File Maintenance: MySQL Server can create a number of different
log files that make it easy to see what is going on. However, you must
clean up these files regularly to ensure that the logs do not take up too
much disk space. When using MySQL with logging enabled, you may
want to back up and remove old log files from time to time and tell
MySQL to start logging to new files.
Unit 14
14.9 Answers
Self Assessment Questions
1. general query
2. --log-output
3. FLUSH
4. host_name.log
5. replication
6. 1 and 10
Terminal Questions
1. The following are various log files available in MySQL:
a. Error Log: Problems encountered starting, running, or stopping
mysqld.
b. General Query Log: Established client connections and statements
received from clients.
c. Binary Log: All statements that change data (also used for
replication).
d. Slow Query Log: All queries that took more than long_query_time
seconds to execute or didn't use indexes.
(Refer Section 14.1)
2. The general query log is a general record of what mysqld is doing. The
server writes information to this log when clients connect or disconnect,
and it logs each SQL statement received from clients. The general query
log can be very useful when you suspect an error in a client and want to
know exactly what the client sent to mysqld.
mysqld writes statements to the query log in the order that it receives
them, which might differ from the order in which they are executed. This
logging order contrasts to the binary log, for which statements are
written after they are executed but before any locks are released. (Also,
the query log contains all statements, whereas the binary log does not
contain statements that only select data.)
Before 5.1.6, the general query log destination is always a file. To
enable the general query log file, use the --log[=file_name] or -l
[file_name] option.
Unit 14
If you specify no filename for the general query log, the default name is
host_name.log in the data directory. If you specify a filename that is not
an absolute pathname, the server writes the file in the data directory.
When --log or -l is specified, the --general_log option also may be given
as of MySQL 5.1.12 to specify the initial general query log state. With no
argument or an argument of 0, the option disables the log. If omitted or
given with an argument of 1, the option enables the log.
(Refer Section 14.3)
3. MySQL Server can create a number of different log files that make it
easy to see what is going on. However, you must clean up these files
regularly to ensure that the logs do not take up too much disk space.
When using MySQL with logging enabled, you may want to back up and
remove old log files from time to time and tell MySQL to start logging to
new files.
On a Linux (Red Hat) installation, you can use the mysql-log-rotate script
for this. If you installed MySQL from an RPM distribution, this script
should have been installed automatically. You should be careful with this
script if you are using the binary log for replication.
You should not remove binary logs until you are certain that their
contents have been processed by all slaves.
On other systems, you must install a short script yourself that you start
from cron (or its equivalent) for handling log files.
For the binary log, you can set the expire_logs_days system variable to
expire binary log files automatically after a given number of days. If you
are using replication, you should set the variable no lower than the
maximum number of days your slaves might lag behind the master.
You can force MySQL to start using new log files by issuing a FLUSH
LOGS statement or executing mysqladmin flush-logs or mysqladmin
refresh.
(Refer Section 14.6)
Unit 14
Reference:
1. MySQL: The Complete Reference by Vikram Vaswani, Tata
McGraw-Hill Publishing Company Limited, Fifth Reprint 2006.
2. MySQL Stored Procedure Programming (Paperback) by Guy
Harrison (Author), Steven Feuerstein (Author); Oreilly Series.
3. MySQL Cookbook (Paperback) by Paul DuBois (Author); Oreilly Series
4. WebSite: http://www.mysql.com