Welcome, Guest. Please login or register.
March 29, 2024, 02:04:09 AM

Home Help Search Calendar Login Register
+  Celtic Woman Forum
|-+  Everything Else
| |-+  Forum Questions & Comments
| | |-+  Why are some characters showing up as diamonds with question marks in them?
« previous next »
Pages: [1] Go Down Print
Author Topic: Why are some characters showing up as diamonds with question marks in them?  (Read 3138 times)
CWazyTom
Global Moderator
Hero Member
*****
Posts: 2095


CWazy for Celtic Woman. Chlovër for life.

View Profile
« on: May 05, 2018, 11:24:15 PM »

Here's a "help" article I posted on some of the other forums in case anyone else is running into this.


Problem:

Some of you (myself included) may have noticed that a few of the characters in posts on these 4 forums show up as diamonds with question marks inside them.

Characters with accents and sometimes characters from text pasted in from another site will show up that way.

The characters will appear correctly when you're posting or editing them and even when you're previewing them.

But when it comes time to view the post in a thread, some characters show up as diamonds with question marks.

It doesn't happen on some of the other forums (Chloë's form and Mairead Nesbitt's forum, for example).

So what's going on?


Cause

I'm still trying to sort out the cause, but it has to do with the way characters are encoded.

You see there are lots of different ways characters can be stored by things like web sites and there are also lots of ways that browsers or other apps can be display those characters.

The diamond with the question mark in it is called a "replacement character" and it's happening because:
- The application / browser trying to display it doesn't know how, OR
- Because the app / browser is trying to display it using a different format than the website is using to provide the character. The forum and browser basically aren't speaking the same language.

In particular, the issue seems to happen when the browser tries to display the characters in the Unicode (UTF-8) format.

Why is this happening for these 4 forums for the current ladies but not some of the other ones?

I'm not entirely sure yet.


What Can I Do About It?

Some browsers allow you to choose the encoding for a web page.

Internet Explorer 11:

- Right click on an empty spot in the web page and click Encoding
- Select something other than Unicode (UTF-8), such as:
* Western European (ISO)
* Western European (Windows)
- Sometimes, choosing Auto-select does the trick

Firefox:

- Press the Alt key if the menu isn't visible yet
- Go to View > Text Encoding
- Select something other than Unicode (UTF-8), such as: Western
- Sometimes, choosing Auto-Detect does the trick

Chrome:

- A while ago, Chrome developers intentionally removed the settings to control the type of encoding to use in your browser to display a web page
- It can apparently still be controlled with a browser extension you can install, but I'm not going to get into that here

Other browsers and mobile devices:

- The option to change the encoding might be different depending on what browser you're using and what type of Operating System or mobile device your're using


I find it annoying, but I can deal with it.

If anyone wants me to look further into it, please post here and I'll see what I can figure out.

Thanks.
Logged
VegasDavid
Sr. Member
****
Posts: 290


🌴

View Profile My home page
« Reply #1 on: August 20, 2018, 12:01:14 PM »

GlenS:  Maybe you could try changing the forum software and database to use UTF-8?  It really is the standard encoding that most new software in the world uses if it cares about special characters, so if we can use UTF-8 on this forum instead of more limited ISO-8859-1 then it should fix a lot of issues.  I still have not figured out how to post Chloe's name with two dots over the e in a way that shows up properly.  While there are workarounds for some browsers like Tom mentioned, it's annoying and needless to force users to switch browsers and employ those workarounds.

For the forum software, there is probably some administrator control panel where you can find a "charset" or "character set" setting and change it to "UTF-8".  Looking through the source code of SMF, it really seems like it does support using UTF-8.

I'd recommend that you also make sure that your database is configured to use UTF-8.  If you are using MySQL, remember to specify utf8mb4, not just utf8.

--David
« Last Edit: August 20, 2018, 12:09:03 PM by VegasDavid » Logged
GlenS
Administrator
Hero Member
*****
Posts: 559


View Profile
« Reply #2 on: August 20, 2018, 07:27:36 PM »

Hey thanks for the info and it looks like I got all the forums switched over to UTF-8 hope it all works

Ok Tom give it a try and see if it works right now
« Last Edit: August 20, 2018, 10:24:52 PM by GlenS » Logged
CWazyTom
Global Moderator
Hero Member
*****
Posts: 2095


CWazy for Celtic Woman. Chlovër for life.

View Profile
« Reply #3 on: August 21, 2018, 07:10:34 AM »

So far it looks great!

Unicode characters are rendering correctly now, even in Chrome.

Thanks!
Logged
VegasDavid
Sr. Member
****
Posts: 290


🌴

View Profile My home page
« Reply #4 on: August 22, 2018, 12:50:32 PM »

Thanks, Glen, that's awesome!

Here are some tests:

Chloë Agnew
Máiréad Nesbitt
😎🍀🎶🎻

The tests were successful.  Thanks!
Logged
GlenS
Administrator
Hero Member
*****
Posts: 559


View Profile
« Reply #5 on: August 22, 2018, 02:04:36 PM »

Glad I could do it. It was a tast the utf-8 files didn't have all the forum icons with it so I had to add all of them to the files to get it to work but after some looking I figured it out :) :)
Logged
Pages: [1] Go Up Print 
« previous next »
 
SMF 2.0.19 | SMF © 2017, Simple Machines
Celtic Woman Forum, Why are some characters showing up as diamonds with question marks in them? - Theme by GlenS