I made a PHP script to export a SQL query (from MySql) to a CSV file for opening in Excel. I cannot control how different users have their Excel setup, ie what separator delimiter they have set (this differs with language settings or user may have changed setting)

So the field separator delimiter gave me some problems, ie Excel would not automatically open file columns correctly but put each row in a single cell, unless you use Import data to make it work.

Some forums suggested setting the separator delimiter in the csv file (delimiter is third argument for fputcsv) to tab (\t) but that did not help.

Solution (use csv header sep=)

Then I stumbled upon the possibility to add a parameter as a header to the actual csv file, simply add sep=<delimiter> to the beginning of your csv file and Excel will parse it. So for separator delimiter use comma (,), pipe (|), tab (\t) or whatever you set in your php code.

However I found a few caveats:

• “sep=\t” needs to be in quotes (otherwise it will use header but also for some reason also show it as data in the spreadsheet and put it in its own cell and move all other cells on that row one position to the right)
• You need to add newline \n after the sep= header (otherwise it will use header but also show it as data concatenated with first cells data)
• Use fwrite as fputcsv will not work

Example to use tab as separator:

[code]fwrite($fp, "\"sep=\t\"\n");[/code]

I could not find much info on the subject of csv headers but this is the full thread that lead me on the to it (thanks Burhan Ali)

Tested with Excel 2016 on Windows 10 and Excel for Mac (15.25) on OSX El Capitan (10.11.15).

I have not tested on other spreadsheets software like OpenOffice, LibreOffice etc, so please share in the comments if you have any success with this.

Now several hours later I have finally figured out what I needed to do…

1) Install the latest version of MySQL Connector/NET. Visual Studio support is from version 6.3 which is currently only available in beta, current version when writing this is 6.3.2 beta.

3) Done!

I am running Windows 7 64-bit, Visual Studio 2010 and MySQL Connector/NET 6.3.2 beta, but according to this thread other people with other versions of VS seems to be having the same problem.

These are the changes that are made to web.config:
[code lang=”xml”]
<connectionStrings>
<remove name="LocalMySqlServer" />
<add name="LocalMySqlServer" connectionString="server=localhost;User Id=root;database=users;password=xxx" providerName="MySql.Data.MySqlClient" />
</connectionStrings>
[/code]

[code lang=”xml”]
<system.web>
<authentication mode="Forms">
<forms loginUrl="~/Account/Login.aspx" timeout="2880" />
</authentication>
<membership defaultProvider="MySQLMembershipProvider">
<providers>
<clear />
<remove name="MySQLMembershipProvider" />
<add name="MySQLMembershipProvider" type="MySql.Web.Security.MySQLMembershipProvider, MySql.Web, Version=6.3.2.0, Culture=neutral, PublicKeyToken=c5687fc88969c44d" applicationName="/" description="MySQL default application" connectionStringName="LocalMySqlServer" writeExceptionsToEventLog="True" autogenerateschema="True" enablePasswordRetrieval="True" enablePasswordReset="True" requiresQuestionAndAnswer="False" requiresUniqueEmail="False" passwordFormat="Clear" maxInvalidPasswordAttempts="5" minRequiredPasswordLength="6" minRequiredNonalphanumericCharacters="1" passwordAttemptWindow="10" passwordStrengthRegularExpression="" />
</providers>
</membership>
<profile defaultProvider="MySQLProfileProvider">
<providers>
<clear />
<remove name="MySQLProfileProvider" />
<add name="MySQLProfileProvider" type="MySql.Web.Profile.MySQLProfileProvider, MySql.Web, Version=6.3.2.0, Culture=neutral, PublicKeyToken=c5687fc88969c44d" applicationName="/" description="" connectionStringName="LocalMySqlServer" writeExceptionsToEventLog="False" autogenerateschema="True" />
</providers>
</profile>
<roleManager enabled="false" defaultProvider="MySQLRoleProvider">
<providers>
<clear />
<remove name="MySQLRoleProvider" />
<add name="MySQLRoleProvider" type="MySql.Web.Security.MySQLRoleProvider, MySql.Web, Version=6.3.2.0, Culture=neutral, PublicKeyToken=c5687fc88969c44d" applicationName="/" description="" connectionStringName="LocalMySqlServer" writeExceptionsToEventLog="True" autogenerateschema="True" />
</providers>
</roleManager>
</system.web>

[/code]

The Workbench edition which includes Query Browser (now called SQL Development) and Administrator (now called Server Administration) is only available in beta but it the old MySQL GUI Tools have already been pulled from Downloads. So I downloaded the latest beta version of MySQL Workbench 5.2.20 and gave it a try.

I’m sorry to say I was very disappointed.

This is what I have found using the SQL Development part (former Query Browser)

1. It is super slow. Doing a query that usually takes a second in the old Query Browser can take minutes! These are just simple queries and it seems the time is not the query itself or time to fetch from server that makes it slow. This alone makes it impossible to use this tool.
2. It is not intuitive. I find it hard to work with, maybe it just takes getting used to
3. Features and functions from the Query Browser are gone, for example in Query Browser it was possible to open up a new tab to run a new query, you could then switch between tabs to see the different results. In the new SQL Development you can open up new query tabs but there is only one results tab. Another thing is that it does not show how many rows were affected by your query. Yet another missing feature is the ease to edit a row, in the old Query Browser all you did was click the Edit button. In the new SQL Development you have to do a specific “EDIT” query (which is even slower than the super slow SELECT) and then go thru several “ok” screens before your edit is finally written to database. I’m also missing is the system Tray Monitor and the Migration Toolkit.

I know it’s only a beta so hopefully they will get these things working in the end but for now I’m going to stick to the old Query Browser.

After some searching I found the old GUI Tools in the MySQL Download Archives.
It seems the latest/last version of GUI Tools was 5.1.12 and can be downloaded here.

The order page on one of my sites have not been working since early November, almost 6 months!! As I’m sure you understand it’s not a very busy site, but usually I get a few orders per month so I should have noticed this a long time ago…

The site is running on IIS/ASP.NET and MySQL server 5.0.45 using the MySQL .NET Connector and I’m using a SQL statement like this on the page:

START TRANSACTION;
INSERT INTO table1 (companyname,street,city,zip) VALUES (‘XS Tech’,’test’,’test’,’12345′);
SELECT @id:=id FROM table1 where id = last_insert_id();
INSERT INTO table2 (id,contactname) VALUES (@id,’Max”);
COMMIT;

This statement was throwing an exception error: “MySql.Data.MySqlClient.MySqlException: Parameter ‘@id’ must be defined”

Since I could not really remember – ie had not properly documented – what changes I may have done to that page six months ago, it made trouble shooting a bit harder. But to make a long story short(er) – It turns out I had upgraded the MySQL .NET Connector and this was causing the error. The site was using MySql.Data.dll version 5.2.1.0. I tried to upgrade to 6.0.3.0 to no avail. But when downgrading to 5.1.4.0 it started to work just fine again.

Finally, I  found that this was indeed a change in the .Net Connector. (I’m not sure what version number but starting from 5.2.0 would be my guess)
Starting from version 5.2.2 of the Connector you should add the Allow User Variables=True Connection String Setting in order to use User Defined Variables in your SQL statements.

Example of Connection String:

Database=testdb;Data Source=localhost;User Id=root;Password=hello;Allow User Variables=True

1. Create the new database, either from MySQL Query Browser (part of MySQL GUI Tools) or MySQL Command Line:

CREATE DATABASE `new_database_name`;

2. Copy the database structure using the mysqldump command from a dos command prompt. Note: If MySQL is not in your path you will have to navigate to the MySQL bin directory, on my machine this is in C:\Program Files\MySQL\MySQL Server 5.1\bin.

mysqldump -d -u root -p -v original_database_name > c:\temp\dbstruct.sql

3. Restore the copied database structure to your new database from the same dos command prompt as in step 2.

mysql.exe -uroot -p new_database_name < c:\temp\dbstruct.sql

Done!

Some notes:
– I used MySQL Server 5.1 when testing this but should work also with earlier versions. If you can or cannot get it to work with another version please add a comment.
– In this example I used root as user but any user with read and create permissions would work.

[Edit: Use mysqldump -R parameter to backup stored procedures and functions, above example would be:
mysqldump -d -u root -p -v -R original_database_name > c:\temp\dbstruct.sql
Link to mysqldump in MySQL Reference Manual ]

BC30560: ‘MySqlConnection’ is ambiguous in the namespace ‘MySql.Data.MySqlClient’.

It worked fine on my development machine but not on the production server. I could not figure out what it was for quite some time. Turned out I had added the mysql.dll to bin directory of my project and at the same time was referencing a shared (GAC) version of the same dll. I had made this and some other changes on the development machine that I had forgot about. The only thing that differed from the development and production machines was the web.config which I obviously did not check well enough.

Posting this here if someonelse or myself makes this stupid mistake again…

To import data from Excel (or any other program that can produce a text file) is very simple using the LOAD DATA command from the MySQL Command prompt.

1. Save your Excel data as a csv file (In Excel 2007 using Save As)
2. Check the saved file using a text editor such as Notepad to see what it actually looks like, i.e. what delimiter was used etc.
3. Start the MySQL Command Prompt (I’m lazy so I usually do this from the MySQL Query Browser – Tools – MySQL Command Line Client to avoid having to enter username and password etc.)
4. Enter this command:

   [code]LOAD DATA LOCAL INFILE ‘C:\\temp\\yourfile.csv’ INTO TABLE database.table FIELDS TERMINATED BY ‘;’ ENCLOSED BY ‘"’ LINES TERMINATED BY ‘\r\n’ (field1, field2);[/code]

5. Done!

Very quick and simple once you know it

Some notes from my own import – may not apply to you if you run a different language version, MySQL version, Excel version etc…

• TERMINATED BY – this is why I included step 2. I thought a csv would default to comma separated but at least in my case semicolon was the deafult
• ENCLOSED BY – my data was not enclosed by anything so I left this as empty string ”
• LINES TERMINATED BY – at first I tried with only ‘\n’ but had to add the ‘\r’ to get rid of a carriage return character being imported into the database
• Also make sure that if you do not import into the primary key field/column that it has auto increment on, otherwhise only the first row will be imported

[Update: I had some trouble getting double quotation marks etc when exporting some Excel data and found a few old but useful Excel export macros in VBA]

[Update: To change the list separator in Excel:
1. First search your Excel file for the character you plan to use so it does not exist somewhere in your data.
2. Go into Windows Control Panel + Regional and Language + Formats Tab click Additional Settings + List separator (Windows 7)]

“C:\Program Files\MySQL\MySQL Server 5.0\bin\mysqldump” –all-databases -u username –password=”password” > c:\backup\%DATE%\Backup_MySQL\backup_all_dbs.sql

A few weeks ago I finally upgraded my MySQL server from 4.1 to 5.0. For once I did everything by the book which included backing up all my databases using the backup tool in MySQL Administrator, installing the new server version and then restoring all databases.

Today however I just realized that somehow two tables in one of the databases did not get backed up and hence not restored. Fortunately I still had the nightly backup from the day before upgrading.

So I thought I’d just do a selective restore from the mysqldump file using MySQL Administrator (part if the MySQL GUI Tools) Turns out MySQL Administrator does not read files created with mysqldump.
And worse: a quick Google reveals that mysqldumps cannot be partially restored

My next couple of hours I spent installing and uninstalling various text editors that could open my 500+MB backup file from mysqldump. I finally found one that could, even though it took a while… (using a Windows Vista machine with 2GB RAM) – NoteTab Light 5.5 (freeware)

Using this I opened the backup file created with mysqldump and sure enough it was a text file containing SQL statements (too bad MySQL Administrator could not open it, hopefully it will in a future version).
I simply deleted the SQL statements for all databases except the one I wanted to partially restore. Then I restored it on my development machine using this command:

“C:\Program Files\MySQL\MySQL Server 5.0\bin\mysql.exe” -uusername -p dbname < c:\backup\2007-12-13\backup_all_dbs.sql

Important note: If you have backed up full databases (not just tables) then it seems the dbname parameter doesn’t matter, it still restores to the original database name – lucky me I did it on my development machine!

After the restore I did a partial backup using MySQL Administrator. Moved that backup file to my production server and did a restore using MySQL Administrator on that machine. Voìla!

So, to sum up:
1) Open your dump file in a large file text editor and delete everything except what you want to restore.
2) On a non-production machine (recommended) do a restore.
3) Make a partial backup using MySQL Administrator from your non-production machine
4) Restore the partial backup using MySQL Administrator on your production server.

Job done, finally I can get back to work!

Of course you can skip step 2 and 3 (and/or just restore the specific tables you want) if you like living dangerously

Happy New Year!

This is when a DATETIME or DATE field contains an invalid value. It turns out that 0000-00-00 which I have used as default value in a DATE column is considered an invalid date. Instead NULL should be used when date/time is not set.

The one solution I found is to add Allow Zero Datetime=True to the connection string, this allows for 0000-00-00 00:00:00 in a DATETIME column. Just add it at the end of your connection string so it looks something like this:
<add key="connectionString" value="Database=your_db;Data Source=localhost;User Id=root;Password=password;Allow Zero Datetime=True;"/>

This however did not seem to work 100% for my code.
In VB.NET I was using this piece of code to check if the valid_until field (of type DATE) is set or not:
If ("" & myDataReader("valid_until")) = "" Then ...

I was concatenating to an empty string istead of using .ToString() because this (if I remember it correctly) would catch both NULL and 0000-00-00 values (both would evaluate to empty string).
Now instead I got this error message: Conversion from type ‘MySqlDateTime’ to type ‘String’ is not valid.

With ODBC, 0000-00-00 used to evaluate to empty string, but using the .NET connector with the connection string work around it returns the actual value, so I have to check for the zeroes instead. So, the solution to my problem is of course to use the proper .ToString function and check for both “0000-00-00” and empty string returned for null value (or even better – use the proper IsDBNull function).

Another way besides using the connection string workaround is of course to fix the data.

• First check the data definitions in your databases to make sure it does not default to 0000-00-00 (for data type DATE) or 0000-00-00 00:00:00 (for data type DATETIME). I’m still running MySQL server version 4 but I think that on version 5, using 0000-00-00 as default value is no longer valid.
• Then update the data. An SQL statement to update a column of type DATE would typically look like this for a column named valid_until:
  UPDATE your_table SET valid_until=null WHERE valid_until LIKE '0000-00-00';

So this post is on how to get it going, it’s very straight forward.
This is tried on Vista with Visual Studio 2005 SP1 and the Visual Studio 2005 Service Pack 1 Update for Windows Vista using the ADO.NET Driver for MySQL (Connector/NET) version 5.0.6.

1) Download and install (using the included installer) the ADO.NET Driver for MySQL Connector/NET.

2) Create or open a project in Visual Studio 2005. I’m using a VB.NET console project in my example.

3) Go to Project + Add Reference…
On the .NET tab find and hilite MySQL.Data and then click Ok.
(It seems this step can be skipped sometimes…)
[Update: In an ASP.NET project go to Website + Add Reference]

4) Go to Project + YourProject Properties… + References Tab
Under Imported Namespaces check MySql.Data.MySqlClient and MySql.Data.Types and save.
[Update: In an ASP.NET project open web.config and add as namespace under configuration – system.web – pages – namespaces]

5) Write your code and run it. Here is an example to get you started:

Dim myConnectionString As String = "Database=TestDB;Data Source=localhost;User Id=TestID;Password=TestPwd"
Dim myConnection As New MySqlConnection(myConnectionString)
Dim myCommand As New MySqlCommand("SELECT * FROM testtable", myConnection)
Dim myDataReader As MySqlDataReader
myConnection.Open()
myDataReader = myCommand.ExecuteReader
While myDataReader.Read
Console.WriteLine(myDataReader("testcolumn"))
End While
myDataReader.Close()
myConnection.Close()

More examples can be found in the manual from MySQL.

Note: If your accessing your MySQL server on another machine, i.e. not as localhost, you need to make sure the firewall on the server allows access on TCP port 3306. If the server is using Windows firewall just simply go into Control Panel – Windows Firewall – Exceptions Tab click Add Port. (Just make sure you remember to disable the port again after you are done testing!)