Alphabetical sorts

[Ron Adam]

I have several applications where I want to sort lists in alphabetical order.
Most examples of sorting usually sort on the ord() order of the character set as
an approximation. But that is not always what you want.

The solution of converting everything to lowercase or uppercase is closer, but
it would be nice if capitalized words come before lowercase words of the same
spellings. And I suspect ord() order may not be correct for some character sets.

So I'm wandering what others have done and weather there is something in the
standard library I haven't found for doing this.

Below is my current way of doing it, but I think it can probably be improved
quite a bit.

This partial solution also allows ignoring leading characters such as spaces,
tabs, and underscores by specifying what not to ignore. So '__ABC__' will be
next to 'ABC'. But this aspect isn't my main concern.

Maybe some sort of alphabetical order string could be easily referenced for
various alphabets instead of having to create them manually?

Also it would be nice if strings with multiple words were ordered correctly.


Cheers,
_Ron



def stripto(s, goodchrs):
""" Removes leading and trailing characters from string s
which are not in the string goodchrs.
"""
badchrs = set(s)
for c in goodchrs:
if c in badchrs:
badchrs.remove( c)
badchrs = ''.join(badchrs )
return s.strip(badchrs )


def alpha_sorted(se q):
""" Sort a list of strings in 123AaBbCc... order.
"""
order = ( '0123456789AaBb CcDdEeFfGgHhIiJ jKkLlMmNn'
'OoPpQqRrSsTtUu VvWwXxYyZz' )
def chr_index(value , sortorder):
""" Make a sortable numeric list
"""
result = []
for c in stripto(value, order):
cindex = sortorder.find( c)
if cindex == -1:
cindex = len(sortorder)+ ord(c)
result.append(c index)
return result

deco = [(chr_index(a, order), a) for a in seq]
deco.sort()
return list(x[1] for x in deco)

[Neil Cerutti]

Re: Alphabetical sorts

On 2006-10-16, Ron Adam <rrr@ronadam.co mwrote:

>
I have several applications where I want to sort lists in
alphabetical order. Most examples of sorting usually sort on
the ord() order of the character set as an approximation. But
that is not always what you want.

Check out strxfrm in the locale module.

>>a = ["Neil", "Cerutti", "neil", "cerutti"]
>>a.sort()
>>a

['Cerutti', 'Neil', 'cerutti', 'neil']

>>import locale
>>locale.setloc ale(locale.LC_A LL, '')

'English_United States.1252'

>>a.sort(key=lo cale.strxfrm)
>>a

['cerutti', 'Cerutti', 'neil', 'Neil']

--
Neil Cerutti

[Tuomas]

Re: Alphabetical sorts


My application needs to handle different language sorts. Do you know a
way to apply strxfrm dynamically i.e. without setting the locale?

Tuomas

Neil Cerutti wrote:

On 2006-10-16, Ron Adam <rrr@ronadam.co mwrote:
>

>>I have several applications where I want to sort lists in
>>alphabetica l order. Most examples of sorting usually sort on
>>the ord() order of the character set as an approximation. But
>>that is not always what you want.

>
>
Check out strxfrm in the locale module.
>
>

>>>>a = ["Neil", "Cerutti", "neil", "cerutti"]
>>>>a.sort()
>>>>a

>
['Cerutti', 'Neil', 'cerutti', 'neil']
>

>>>>import locale
>>>>locale.setl ocale(locale.LC _ALL, '')

>
'English_United States.1252'
>

>>>>a.sort(key= locale.strxfrm)
>>>>a

>
['cerutti', 'Cerutti', 'neil', 'Neil']
>

[Leo Kislov]

Re: Alphabetical sorts



On Oct 16, 2:39 pm, Tuomas <tuomas.vesteri ...@pp.inet.fiw rote:

My application needs to handle different language sorts. Do you know a
way to apply strxfrm dynamically i.e. without setting the locale?

Collation is almost always locale dependant. So you have to set locale.
One day I needed collation that worked on Windows and Linux. It's not
that polished and not that tested but it worked for me:

import locale, os, codecs

current_encodin g = 'ascii'
current_locale = ''

def get_collate_enc oding(s):
'''Grab character encoding from locale name'''
split_name = s.split('.')
if len(split_name) != 2:
return 'ascii'
encoding = split_name[1]
if os.name == "nt":
encoding = 'cp' + encoding
try:
codecs.lookup(e ncoding)
return encoding
except LookupError:
return 'ascii'

def setup_locale(lo cale_name):
'''Switch to new collation locale or do nothing if locale
is the same'''
global current_locale, current_encodin g
if current_locale == locale_name:
return
current_encodin g = get_collate_enc oding(
locale.setlocal e(locale.LC_COL LATE, locale_name))
current_locale = locale_name

def collate_key(s):
'''Return collation weight of a string'''
return locale.strxfrm( s.encode(curren t_encoding, 'ignore'))

def collate(lst, locale_name):
'''Sort a list of unicode strings according to locale rules.
Locale is specified as 2 letter code'''
setup_locale(lo cale_name)
return sorted(lst, key = collate_key)


words = u'c ch f'.split()
print ' '.join(collate( words, 'en'))
print ' '.join(collate( words, 'cz'))

Prints:

c ch f
c f ch

[Ron Adam]

Re: Alphabetical sorts

Neil Cerutti wrote:

On 2006-10-16, Ron Adam <rrr@ronadam.co mwrote:

>I have several applications where I want to sort lists in
>alphabetical order. Most examples of sorting usually sort on
>the ord() order of the character set as an approximation. But
>that is not always what you want.

>
Check out strxfrm in the locale module.
>

>>>a = ["Neil", "Cerutti", "neil", "cerutti"]
>>>a.sort()
>>>a

['Cerutti', 'Neil', 'cerutti', 'neil']

>>>import locale
>>>locale.setlo cale(locale.LC_ ALL, '')

'English_United States.1252'

>>>a.sort(key=l ocale.strxfrm)
>>>a

['cerutti', 'Cerutti', 'neil', 'Neil']

Thanks, that helps.

The documentation for local.strxfrm() certainly could be more complete. And the
name isn't intuitive at all. It also coorisponds to the C funciton for
translating strings which isn't the same thing.

For that matter locale.strcoll( ) isn't documented any better.



I see this is actually a very complex subject. A littler searching, found the
following link on Wikipedia.

http://en.wikipedia.org/wiki/Alphabetical_order#Compound_words_and_special_characters

And from there a very informative report:

http://www.unicode.org/unicode/reports/tr10/

It looks to me this would be a good candidate for a configurable class.
Something preferably in the string module where it could be found easier.

Is there anyway to change the behavior of strxfrm or strcoll? For example have
caps before lowercase, instead of after?


Cheers,
Ron

[Neil Cerutti]

Re: Alphabetical sorts

On 2006-10-17, Ron Adam <rrr@ronadam.co mwrote:

Neil Cerutti wrote:

>On 2006-10-16, Ron Adam <rrr@ronadam.co mwrote:

>>I have several applications where I want to sort lists in
>>alphabetica l order. Most examples of sorting usually sort on
>>the ord() order of the character set as an approximation.
>>But that is not always what you want.

>>
>Check out strxfrm in the locale module.

>
It looks to me this would be a good candidate for a
configurable class. Something preferably in the string module
where it could be found easier.
>
Is there anyway to change the behavior of strxfrm or strcoll?
For example have caps before lowercase, instead of after?

You can probably get away with writing a strxfrm function that
spits out numbers that fit your definition of sorting.

--
Neil Cerutti
Whenever I see a homeless guy, I always run back and give him
money, because I think: Oh my God, what if that was Jesus?
--Pamela Anderson

[Jorgen Grahn]

Re: Alphabetical sorts

On Mon, 16 Oct 2006 22:22:47 -0500, Ron Adam <rrr@ronadam.co mwrote:
....

I see this is actually a very complex subject.

....

It looks to me this would be a good candidate for a configurable class.
Something preferably in the string module where it could be found easier.

/And/ choosing a locale shouldn't mean changing a process-global state.
Sometimes you want to perform something locale-depending in locale A,
followed by doing it in locale B. Switching locales today takes time and has
the same problems as global variables (unless there is another interface I
am not aware of).

But I suspect that is already a well-known problem.

/Jorgen

--
// Jorgen Grahn <grahn@ Ph'nglui mglw'nafh Cthulhu
\X/ snipabacken.dyn dns.org R'lyeh wgah'nagl fhtagn!

[Ron Adam]

Re: Alphabetical sorts

Neil Cerutti wrote:

On 2006-10-17, Ron Adam <rrr@ronadam.co mwrote:

>Neil Cerutti wrote:

>>On 2006-10-16, Ron Adam <rrr@ronadam.co mwrote:
>>>I have several applications where I want to sort lists in
>>>alphabetic al order. Most examples of sorting usually sort on
>>>the ord() order of the character set as an approximation.
>>>But that is not always what you want.
>>Check out strxfrm in the locale module.

>It looks to me this would be a good candidate for a
>configurable class. Something preferably in the string module
>where it could be found easier.
>>
>Is there anyway to change the behavior of strxfrm or strcoll?
>For example have caps before lowercase, instead of after?

>
You can probably get away with writing a strxfrm function that
spits out numbers that fit your definition of sorting.

Since that function is 'C' coded in the builtin _locale, it can't be modified by
python code.

Looking around some more I found the documentation for the corresponding C
functions and data structures. It looks like python may just wrap these.

http://opengroup.org/onlinepubs/0079...bd/locale.html


Here's one example of how to rewrite the Unicode collate in python.

http://jtauber.com/blog/2006/01

I haven't tried changing it's behavior, but I did notice it treats words with
hyphen in them differently than strxfrm.



Here's one way to change caps order.

a = ["Neil", "Cerutti", "neil", "cerutti"]

locale.setlocal e(locale.LC_ALL , '')
tmp = [x.swapcase() for x in a]
tmp.sort(key=lo cale.strxfrm)
tmp = [x.swapcase() for x in tmp]
print tmp


['Cerutti', 'cerutti', 'Neil', 'neil']



Cheers,
Ron