PK JournalComputers

ChatGPT: Testing out the rumors

ChatGPT is a chat service with an AI bot. It is fed many samples of text from all over the net, and can answer your questions in a conversational style. I have seen videos that uncharatably characterizes chatGPT as quite often giving wrong answers to questions. Academics even play down the bot’s ability to give coherent answers.

Well, I can tell you that despite it being tied to corpus linguistics and samples given to it, chatGPT seems to learn how to respond pretty well.

There was once a famous phrase that is touted to be correct grammar but still completely nonsensical: green dreams sleep furiously. So I asked chatGPT:

why do green dreams sleep furiously?

ChatGPT: I’m sorry, but I’m not sure what you mean by “green dreams sleep furiously.” This sentence appears to be a nonsensical combination of words that does not convey a clear or meaningful message. If you could provide me with more context or rephrase your question, I would be happy to try to help you to the best of my abilities.

So it wasn’t fooled by this standard phrase found in many texts on linguistics. I thought it was entirely possible that this phrase would be so well known to the designers, that it was fed to ChatGPT as a “bad” example of a meaningful phrase.

But then I remembered a video of a prof asking chatGPT about blue wolves, and apparently, chatGPT took him seriously and attempted a serious answer using reasoning that was unsupportable on a casual Googling. There are no blue wolves. So I tried this:

can you tell me about green foxes?

ChatGPT: In the natural world, foxes come in a variety of colors, including red, gray, and even black. However, there are no known species of foxes that are naturally green.

That being said, in some fictional and mythological contexts, green foxes do appear. For example, in Japanese folklore, there is a creature called the “Kitsune” which is a fox spirit with the ability to shape-shift into a human form. Kitsune are often depicted as having red or white fur, but some stories describe them as having green fur.

In modern media, green foxes also appear in various forms of art and entertainment, such as video games and cartoons. For example, in the popular video game series “Star Fox,” the main character, Fox McCloud, is depicted with green fur.

Overall, while there are no known naturally occurring green foxes, they do have a presence in various fictional and artistic contexts.

I couldn’t fool it with that either. I am now starting to chalk it up to maybe a peculiar phrasing the prof was using, which I didn’t have the knack for. Fooling the bot isn’t easy then, and you have to know its weaknesses in a way that a layperson like me is not likely to grasp. Clearly, it is not just riffing “found text” together as claimed in any kind of haphazard manner.

But as a programmer, common sense tells me that a computer isn’t going to know the difference between good and bad responses to samples. It also isn’t going to predict, or have stored answers to, your questions. And indeed, each response to chatGPT allows me to rate an answer by clicking one of two icons — either a “thumbs up” icon or a “thumbs down”.

It is likely after millions of questions from as many users, the sampling would be far richer than what the designers alone would have given.

I even tried to deliberately trip it up with a question like “Why was president Jello Biafra considered an anarchist?” The question is premised on the Dead Kennedys former frontman being “president”, which is untrue. Here is its response:

Why was president Jello Biafra considered an anarchist?

ChatGPT: Jello Biafra, whose real name is Eric Reed Boucher, is an American musician and political activist who is best known as the lead singer of the punk rock band “Dead Kennedys.” He is often associated with anarchist and leftist politics.

Biafra was heavily influenced by the punk rock movement of the late 1970s and early 1980s, which was characterized by a DIY ethos and a rejection of mainstream culture and politics. He was particularly influenced by the British punk band “Crass,” who espoused anarchist and pacifist ideas.

Biafra’s lyrics and political activism often expressed anarchist and leftist ideas, such as opposition to corporate greed, government corruption, and police brutality. He also supported a number of progressive causes, such as environmentalism, animal rights, and LGBTQ rights.

While Biafra himself has never identified as a strict anarchist, his political beliefs and activism have often aligned with anarchist principles of anti-authoritarianism, direct action, and solidarity with marginalized groups. His advocacy for alternative forms of social organization and direct democracy have also been associated with anarchist thought.

Overall, Biafra’s anarchist and leftist leanings are a result of his political and cultural influences, as well as his own experiences and observations of social and political issues in the United States.

It seems to know its counterculture well enough to fool me, a former college DJ (except on some minor points, like his being influenced by the British punk band “Crass”, which I never heard about in all of my years following his biography since the 1980s). But it ignored my reference to him as “president”. Other things that seem just a bit “off”: I don’t think he was ever vocal on animal or LGBT rights (although I am sure he didn’t oppose them), so these are also unsupported.

When I tried a more direct question, “What year was Jello Biafra elected president?”, it then rejected my premise that he was ever president or indeed had ever held political office except in a satirical scenario. Damn. Nailed him.

Then I asked it a Calculus question:

What is the tangent equation of the function x^2 at the point (2.4)?

ChatGPT: To find the equation of the tangent line to the function f(x) = x^2 at the point (2,4), we need to find the slope of the tangent line at that point, and then use the point-slope form of a line to write the equation.

The slope of the tangent line to f(x) at x = 2 is equal to the derivative of f(x) at x = 2. We can find this derivative using the power rule of differentiation:

f'(x) = 2x

So, at x = 2, we have:

f'(2) = 2(2) = 4

This means that the slope of the tangent line to f(x) at x = 2 is 4.

Now we can use the point-slope form of a line to write the equation of the tangent line. The point-slope form is:

y – y1 = m(x – x1)

where m is the slope of the line and (x1, y1) is a point on the line.

Plugging in the values we found, we have:

y – 4 = 4(x – 2)

Simplifying, we get:

y – 4 = 4x – 8

y = 4x – 4

So the equation of the tangent line to f(x) = x^2 at the point (2,4) is y = 4x – 4.

That was scary. Notice that the question was asked with a typographical error: (2.4) should have been (2,4) in the question, but chatGPT just took it all in stride, corrected my mistake and answered the question. It didn’t even point out the mistake, it just dealt with it quietly, as if it was respecting my feelings or something.

The prospect of education and chatGPT

I can say quite succinctly that with chatGPT’s ability to produce essays or to solve math problems with detailed descriptions of the steps that education is clearly under threat.

I also asked chatGPT to produce Java code of the first 10 Fibonacci numbers along with ratios of the successive pairs of terms. It produced the Java code, which printed the terms and ratios on the same line, which I would consider the only bug in the program, albeit a minor bug. It also added comments in key places int he code. It only took a few changes of their print() statements into printf() statements before the output became presentable. Code formatting was excellent. But here I take a grade 10/grade 11 problem in programming, and it appears a call to chatGPT can now solve those programming problems, and very likely more complex problems as well. And do it in seconds.

The prospect of love, relationships and chatGPT

I love my wife, I really do. And I’m a lousy poet. So can chatGPT write romantic sonnets? Oh yeah. You pick the flavour: standard, Petrarchean, Shakespearean — those were the styles I’ve tried. The first one was the one I finally gave to my wife Diane as a “found poem with her name by an unknown author”. While that is true in its content — the AI bot just curates samples of text from all over the ‘net and munges it all together in interesting and new ways, so indeed attribution is impossible — it was the standard “white lie” of a husband who had almost forgotten today was Valentine’s day, and who had not the heart to tell her that the sonnet was computer-generated through artificial intelligence.

But I meant every word of the poem.

Write a sonnet for my wife diane

ChatGPT: Of all the things that I have come to know,
There’s none so fair or dear as Diane’s face.
Her eyes, like pools of light, with love aglow,
Her smile, a thing of wonder and of grace.

She moves with such a poise and gentle ease,
A joy to see her dancing through the day.
Her laughter, like a bird upon the breeze,
Can lift me up and chase my cares away.

In times of need, she is my shining light,
A beacon in the darkness of my soul.
Her love has helped me through the darkest night,
And made me once again feel whole.

Diane, my love, my heart’s own true delight,
You are the sun that makes my world so bright.

The most “popular” programming languages since 1960

There will be people that will dismiss any “popular” programming list as a kind of meaningless horse race. But that’s as long as you are not looking for job prospects as an answer to what will pay the bills and keep the lights on when you enter the work world. But the field is large enough that you have room to ask yourself: what kind of programming do you want to do? Systems programming? Applications? Servers? Clients? Scientific models? Statistical studies? Device drivers? Everyone hears about web programming, since that is the most visible, and seems to get the most “airplay” in the media. It might even interest you. For others, it’s dull. There is so much more out there.

With that preamble, why am I bothering to still do this? It is to show how popular languages follow the ebb and flow of computing history. Since World War II, we had the ENIAC, a host of IBM and AT&T mainframes, followed by networked computers, then personal computers, then the internet, and so on. With each major shake-up, programming needs change.

By 1965, what had changed preferences in computer languages, are the same things that change it today: changes in hardware, programming for mainframes versus “personal” computers (which in this decade amounted to comptuers like the PDP-1). In the 1960s, hard drives (which were called “disk drums” back then) were relatively new, as was magnetic tape. Transistors hadn’t quite made their heyday yet, with the some of the most powerful computers still using vacuum tubes.

1960

COBOL. 1960 saw the introduction of supercomputers in the service of business, and by far the most popular language was COBOL (COmmon Buisiness-Oriented Language). COBOL was an interpreted language, which meant it was capable of running on many different machines with a minimum number of changes. Today, by the end of 2022, over 80% of business code is still written in COBOL.

1965

The Olivetti programmable calculator, about the size of a small modern digital cash register, and among the first of its kind.

ALGOL. Algol-60 saw the first implementation of the QuickSort algorithm, invented by C. A. R. Hoare, while a student in Moscow. He was knighted by Queen Elizabeth II for his discovery. Algol was behind COBOL as the most popular programming language, but both were dwarfed by FORTRAN users.

FORTRAN. FORTRAN was far and away the most popular programming language by 1965, and stayed that way for some decades. It was taught in many “service” computer course taken by science students and most engineering students. It was known for having a rather elaborate mathematics capability.

Other languages popular during that period: Assembly, APL, BASIC and Lisp. 1969 was the year that PASCAL was first introduced, by Niklaus Wirth.

1970

1970 saw the invention of UNIX by Kernighan and Ritchie at AT&T Labs, and Pascal came on board as a teaching language for structured programming in many university freshman courses. Otherwise, the landscape was pretty much the same for programming languages in popular use as before.

1975

By 1975, C had grown in popularity, but was not a teaching language: BASIC, Pascal, and Lisp had all ascended in popularity as we had sent men on the moon, and more students became interested in computer programming. FORTRAN and COBOL were still at the top of the heap, while ALGOL, APL and Assembly moved down. Assembly would in future decades disappear from general popularity, but it would never truly go away.

1980

By 1980, C++ had been introduced by Bjarne Stroustrup over the past couple of years, bringing the concept of object-oriented programming to the world. More and more people had mastered C, and it moved to the middle of the “top 10” proramming languages used that year. Pascal became a wildly more popular language due to the introduction of household desktop PCs, and the offering of a Turbo Pascal compiler by a software company called Borland. Microsoft offered BASIC and FORTRAN compilers that extended their stock QBASIC interpreter that came with DOS. In addition, Tandy, Commodore and Sinclair were offering their own machines, each with their own BASIC interpreters.

1985

While he didn’t invent the Internet (he never claimed that at all, according to Snopes.com), Al Gore tables bills and sources funding to greatly expand the internet, post 1989.

Bjarne Stroustrup publishes his seminal work The C++ Programming Language, in 1985. With the introduction of Windows and Windows NT, Microsoft expanded their programming offering to include Visual Studio, which included compilers for C and C++. C was rising to the top of the charts, competing with Borland’s Pascal product. C would never leave the top 3 for another 15 years.

1990

Visual BASIC was introduced by Microsoft. C++ rose to the top 5. FORTRAN, BASIC, Assembly, and COBOL all fell to the bottom 5 of the top 10 languages. C had a wild surge in popularity, as the Internet was coming onstream, and the World-Wide Web was just starting in the universities. By 1992, the top 2 positions were occupied by C and C++. Also by 1992, a need for CGI scripting was needed for the fledgling W0rld-wide web, and Perl became popular.

1995

There were many scripting languages at the time aimed at web browsers, but there had not been any set standard as to a default scripting language. By the end of the decade, that standard would go to JavaScript, a language developed since 1995. It and Perl were rising in popularity as client-side and server-side web-based languages respectively. But in the following 5-year period there was another shake-up. Java (a very different language from JavaScript), a product of Sun Microsystems, came from out of nowhere in 1995 to be the 3rd most popular language by 1996. By this time, the web had arrived in people’s homes and there was a need to enhance people’s internet experiences.

Pascal was falling out of favour as computers were moving away from DOS in the home and in business, and by 1997, Borland designed and object-oriented version of Pascal, which was called Delphi. It turned out to be a formidable competitor to Visual Basic. By 1998, even more server-side dynamic web programming was provided with the language PHP.

2000

PHP overtook Perl by 2000 as the 5th-most used language that year. Java and JavaScript had occupied 2nd and 3rd, pusing C++ to the #4 spot. C was still on top. That year, Microsoft offered the world C#. Apart from C and C++, the top 5 langugaes were all web-based languages: Java, JavaScript and PHP. Perl was descending in popularity, as a new scripted language with much cleaner syntax became ascendant: Python.

2005

C was finally pushed out of the top spot by Java; and Delphi was starting to drop out of the picture as Borland had financial troubles after a failed bid to attempt to make inroads into Linux, with their introduction of Kylix. They sold off Delphi to Embracadero, who produces that product today. Perl continues to ascend in popularity only slowly, as its popularity is buoyed up by a legacy of libraries and its role in various bioinformatics projects, such as the Human Genome Project, conducted by universities around the world.

In part due to bioinformatics and other informatics endeavours, math and stats-based languages popped up such as Matlab and R. There were still new web-based languages like Ruby.

2010

Perl had finally dropped off the top-10, leaving a legacy of code on web servers all over the world. Objective-C became popular with Apple developers and new operating systems line NextStep, iOS and OS X. By 2011, the top 4 were: Java, JavaScript, Python, and PHP. Apple’s teaching language, Swift was at #9 in 2014.

2015

C and C++ were pushed out of the top 5. R, primarily a statistical programming language, rose to #7, second only to C. By 2019, Python was the top language programmers were using. Kotlin showed up briefly in 2019, owing to Google’s support of the language on the Android.

2020

Not much change, except for the introduction of Go, touted to be a more “reasonable” implementation of C++ with lighter syntax. Microsoft introduced TypeScript, a superset of JavaScript, and likely an attempt to “embrace and extend” it as they attempted to do the last time to Java, for example (J++ never caught on), or to JavaScript itself with their mildly successful VBScript, which also never quite caught on over the long haul.

While that was happening, Rust, which had been around for some time, enjoyed some popularity as a back-end web language, as well as a systems language. By the end of 2022, TypeScript has risen to the top 5. Of 11 languages that are the most popular, 7 are web-based languages: Python, JavaScript, TypeScript, PHP, Go, Rust, and Kotlin. The others are Java, C++, C, and C#.

Ideology vs Getting Stuff Done

I’ve disliked ideology as a personal philosophy. Nothing ever really applies in all cases. This is why, while I think of the idea of open source software is awesome, it is not awesome if you have to sacrifice doing things you need to do in the name of philosophical purity.

I share the views stated by Evan Liebovich in a video he made last year. He is more of a proponent of open source than I will ever be, but he has had to reckon with the fact that Linux is falling behind on the desktop, is a low priority for developers, and if you actually need to get things done for many ordinary use cases, you need to install Windows. Evan’s video was shot from his Windows 10 desktop.

Linux will still be the low-cost desktop solution for developers, sporting a plethora of sophisticated programming tools. In that one area, they are way ahead of all other operating systems. However, in other respects for example, Microsoft can support the latest scanners and media cards, which Linux is often slow in adopting.

The reason they are slow is, as of 2021, according to Evan, Linux is installed in only 1.8% of all desktops. This ought to be regarded as minor, considering Linux’s unquestioned dominance in android devices, Chromebooks, business infrastructure, and internet servers all over the world. Linux has scored dominance in nearly every platform imaginable. Just not on the desktop PC.

People writing software and drivers for the Windows PC are likely to stay programming in that domain, since that is where 98% of the market is. It is not very likely that most companies, especially small ones, are going to write drivers for hardware or peripherals such as adaper cards, printers or scanners since there is not enough money in it. But even if they do, it is not as likely to fully take advantage of the hardware.

Nowadays you can download licences for Windows 10 which had been taken from old, discarded machines. This is legal if a legitimately purchased licence was on one machine, is no longer used there, and is transferred to a new machine. Microsoft will have a problem if one license is on more than one machine. Otherwise, there should be no problem. Ali Express sells licences for as low as CAN$3.50 by some vendors. Some sell with DVD, others sell only the license code. In the latter case, you can go to Microsoft.com and download your own image to a USB stick, 16GB minimum.

Xemacs, xcalc, xclock, and TeXStudio, all running on Windows 10 using an X server called “vcxsrv”, all thanks to WSL2.

Microsoft has, in the view of many, moved on from the “OS wars”, and have allowed users to incorporate WSL2 (Windows Subsystem for Linux version 2) into their base operating system. This has allowed users to run Linux applications, and even X-Windows applications, on their Windows desktop. Another gesture to open source is their purchase of GitHub, and have also joined the Linux Foundation. MS Teams also has a version made for Linux.

Even without WSL2, there are many open source (FOSS) applications that have windows versions. And chances are, they run better in Windows, since writers of video drivers, for example, likely have better support of graphics acceleration. The same can also be said for audio, printer support, network card support and scanner support. Such FOSS applications which do not use WSL2 can be found on sites such as TTCS OSSWIN Online (Trinidad and Tobago Computer Society collection of free Open Source Software for WINdows), which has one of the most comprehensive FOSS archives for MS Windows.

It is still possible, by and large, to have a FOSS computer where the only “large” software expense is the Windows OS itself.

What is that sorting algorithm?!

Yes, for a while I was mystified. I had taught what I was sure was the Bubble Sort, the most basic sorting algorithm given to computer students. It was so simple, I felt little need to look it up (but should have), and just write it from memory. Here was my algorithm, written in pseudocode:

You have an array of numbers A[]
for i <- 1 to N
   for j <- (i+1) to N
      if A[i] > A[j] then
         swap A[i] and A[j]

That’s the Bubble Sort, as I thought it was, and as I have been teaching it for 5 years. The algorithm works, is inefficient as the Bubble Sort has been advertised to be, and I have never questioned it. It was also an easy one to teach and students picked up on it easily.

But from this or that source over the 5 years, I have found others attributing a wholly different algorithm to the Bubble Sort. I dove into my copy of Algorithms 2e (Cormen, Leiserson, RIvest and Stein, 2001), a text which was used in many of the better computer science programs in North America, and this was how they described it:

Let A be an array of data to sort
for i <- 1 to length[A] do
   for j <- length[A] downto (i + 1) do
      if A[j] < A[j - 1] then
         Swap A[j] and A[i]

Yes, this is the source I should have cited rather than going on memory, and that would have been the end of it. But notice that we are only ever comparing two adjacent array elements in a bubble sort. But at the time, I only saw evidence of this cropping up on a YouTube video and some other nameless, faceless websites, and thought maybe there is more than one algorithm called a bubble sort. I tried this several times on paper with several short combinations of numbers, and convinced myself that their algorithm works also.

There are other versions. Harvard CS50 adds some efficiency to the pseudocode. Since the list is often sorted well before both loops have completed, the modification is to have a conditional loop (rather than a counted one) which uses a swap counter, which is used to decide if the list really is sorted. If there are no swaps after one turn of the outermost loop, the list is declared sorted and the algorithm exits. It went something like this:

Let A be an array
Set swap_counter to a nonzero value
Repeat
|   reset swap_counter to 0
|   look at each adjacent pair in A
|   |  if A[n] and A[n-1] are not in order,
|   |     Swap A[n] and A[n-1]
|   |     Add 1 to swap_counter
|   v  end if
v   move to the next adjacent pair
Until swap_counter is 0

You have to play with some short lists on a sheet of paper to convince yourself that this will actually sort a list. It is still a nested loop, but a conditional one.

Then, there was Wikipedia. Their entry on Bubble Sort, apart from being about the proper bubble sort, had a footnote on the bottom, to a paper describing a sorting method called I can’t believe it can sort. Seriously, that was the name given and cited. Here is their algorithm, as elucidated by Dr. Stanley Fung from the University of Leicester (UK) (3 October, 2021) in a paper he entitles “Is this the simplest (and most suprising) sorting algorithm ever?”, publically archived at Cornell Univesity:

Let A be an array of length n
for i <- 1 to n do
   for j <- 1 to n do
      if A[i] < A[j] then
         Swap A[i] and A[j]

This is where I began to question my sanity. This was very close to my version of Bubble Sort, except that I started $j$ at $(i+1)$ and tested $A[i] > A[j]$ . It was similar in that it used a nested for loop, comparison, and swap.

But Fung wrote a paper giving this as a kind of Ripleys’ Believe it or Not. He seemed incredulous that this could work at all. And yes I can see his point. His test of $A[i] < A[j]$ seems backward, yet the numbers end up in increasing order. This is becuase allowing both nested loops going to the array size allows for redundant comparisons and for additional swaps which can take place if $j < i$ , which he explains. Is it efficient? Hell, no. But it is still shares a similar efficiency to Bubble Sort, $O(n^2)$ . Would anyone in their right mind offer this to students as a sorting algorithm? Nope. And yes, he does make these points also. It’s as close as I have seen to a sorting algorithm that appears “brute force”. In addition, he gives a proof of correctness of the algorithm to sort in ascending order.

But now I began to wonder what I was doing. I had been using something like this algorithm for 4 years before Fung wrote his paper. What was my algorithm then? Did I invent something new? Am I going to be famous? That feeling was short-lived as I read on. The next algorithm he offered was the one I taught. It was called an Exchange Sort.

I stand by Exchange Sort as an basic algorithm to teach and for students to grasp in high school, and seems more straightforward than the Bubble Sort. I will probably end up teaching both in due course, and I have already changed my notes and slides, as I haven’t taught it yet this year.

This whole episode illustrates the importance of readily letting go of old ideas once they have been proven wrong, and then learning the right way. Learning takes humility, one of our human virtues we are endowed with. People who feel no need of such reflection and change are people whom I feel are crippled with one of the worst learning disabilities, not the result of biological barriers or mental illness, but instead merely borne of pride and hubris. Life is too short to hang on to old ideas that don’t work and actually never did in the past either.

The Virtual Existence of Advameg

Advameg.org is a toy of Lech Mazur. The Illinios-based website is a throwback to “World-Wide-Web one-point-oh”, which has recently had some CSS3 applied to it. But, it still doesn’t take away from the underlying design of the site, which are rooted in a bygone web architecture. One person said the site looked more like throwback from the days of Windows 95.

Many parts of the site appear to be somewhat up-to-date, due to their pulling of public data from whatever outside agencies collect them and make free and public. I have noticed that their RFCs are also pretty much up-to-date also. When I say “up-to-date”, is that everywhere I looked, the latest data, updates, and RFCs are from 2019. To me, this is much more up-to-date than expected (but still not really). All of this free data is curated over 72 of their websites such as City-data, Nonprofit Facts, Photo Dictionary, and one website with the ambitious name “Website Encyclopedia“. There is probably nothing in these sites that you couldn’t look up with Google and get more updated and more reliable information. It would also be naive to think that all 200 million websites and 400 million registered domains which are currently active would be listed under Website Encyclopedia. Pile on to that the fact that there are a half million websites being added to their number each day.

What has not kept up with the times is their FAQ archive. It is a graveyard of FAQs that have not been updated in almost 20 years. This could once have been easily updated by checking the MIT FAQ archive. But since that is no longer in existence and USENET is pretty much dead in North America, most FAQ maintainers have found better ways of making their FAQs public, usually through making a web page. The founder of Advameg, Lech Mazur, was elusive and seemingly impossible to reach even prior to 2010, and by all accounts, even more difficult to reach today. It is one thing to remove a FAQ that is no longer topical or has links to other internet sites that are no longer working; it is another thing if you once posted opnions you thought were worthwhile 20 years ago, but no longer support.

As a bit of advice, if you see a FAQ title in faqs.org that interests you, try Googling the same title. Chances are you will get a more updated FAQ by the same author(s).

A website called Ripoff Report indicates that they have resisted requests to update or remove information that is no longer useable, sometimes in a way that appears intimidating, or in a way that uses their access to information about the complainants to reveal names, addresses and identities of anyone who complains about their services.

Also, it appears as though even a 200-word Wikipedia article about city-data.com was unusually difficult to maintain. There were vandals who posted content on the article accusing the discussion board of tolerating racism (this content was quickly taken down). In the Talk section there was extensive discussion about the site, including one posting in 2015 from an Lech Mazur himself who went on in excess of 500 words (more then double the word count of the original article itself) in great detail how to “bring the article up to Wikipedia standards”. He seemed mostly to be self-advocating, rather than bending Wikipedia to his will. He was posting his views on how it should be written, and the Wikipedia writers responded with their own review of how articles get written and sourced.

city-data.com has been cited a few times over the years by other websites like the New York Times as a reference to stats about things like housing prices, or in one case, their city-data discussion forum has been mentioned once when the topic of commuting from New York City to Scranton, Pennsylvania (120 miles away) came up. There was no comment on this, just a link. Therefore, it is puzzling to stumble into the review of this site from complainants to the Better Business Bureau, with various kinds of racism. The kinds appear inconsistent, with some complaining of the site becoming a hotbed for hate groups and trolls of various persuasions. The data side of this site is out of date by several years, and not that useable, as many other commenters to the BBB had noticed.

It looks as though, apart from being on LinkedIn these days, Mazur is also posting tweets about tech and science on Twitter. He is trying to position his company, it appears, in the area of artificial intelligence and natural language processing. He does not accept tech support questions on his Twitter feed. I would guess that also means no complaints about city-data or faqs.org.

A Pet Peeve About the Internet

I have said this several times before in many ways, and I will say it again: there is too much importance placed on the internet. It wouldn’t be so bad if the internet was run by the government, since that would make it more accountable. But instead, it is mostly in the hands of large private companies, who are largely unaccountable, and would not be truthful unless regulators, or the threat of regulation, forces their hand.

This was brought out in all its glory yesterday, as the Canadian telecommunications conglomerate, Rogers, experienced a denial of service nationwide, affecting all internet services. It wasn’t just that families were denied Netflix or YouTube, or that you couldn’t receive email or text messages, it was that the entire economy slowed considerably. Interac stopped working, and that meant that people couldn’t make transactions unless they had cash or credit. All major banks and credit unions use Interac, and thus experienced this problem. But in addition, customers were also not able to do e-transfers or pay bills through their bank.

Many vending machines are hooked up to the internet, and some of them were disabled if they had to connect with a Rogers service. Municipal parking, now dependent on the internet as many cities abandon parking meters, were hobbled as municipalities were not able to accept payment for parking. Toronto’s BikeShareTO service, whcih depends on the internet to distribute bikes to users, had to declare their bikes inaccessible for all stations in Toronto. Vancouver had a similar problem with its bike share program. School boards with summer online programs had to go asynchronous and move its deadlines back another day. Public libraries had a stoppage of WiFi service at many locations, as well as self-checkout machines, and book kiosks.

Many retail stores had to close altogether for the day, as was seen in Mississauga’s Square One Mall and Toronto’s Yorkdale Mall. Many condos and apartments experienced a disabling of their buzzer systems due to the outage.

That wasn’t all. 911 services also stopped working in many areas where the 911 services were managed by Rogers. Chatr Mobile and Fido, offshoot services owned by Rogers, also stopped working. Downstream internet service providers (ISPs) also experienced downtime as a consequence.

It caused the phone lines at the CRTC to go down, since they were using IP telephony provided by Rogers. Of course, this would also be true for any IP telephone, which includes all cell phones served by Rogers or its subsidiaries. These IP telephones also went down in passport offices provided by Service Canada. Rogers also manages the multi-factor authentication systems used by the Canada Revenue Agency, so anyone attempting to log in to the CRA website yesterday could not log in.

One way the internet is oversold is in how we market and use IoT (Interent of Things) devices. Imagine for example, the many forms of digital signage you see around you. A good number of them are connected to the internet, and programmed remotely. Examples are digital highway signs which warn of dangers ahead. So are a good number of household appliances, including televisions and tablets. IoT can also centrally link home security systems, allowing a monotoring service to provide surveillance at low cost as if a security guard was on site. IoT has medical applications, such as providing a way to aid medical professionals to monitor someone who has cardiovascular disease. Some of these things seem pretty essential, but others, such as IoT stoves or refrigerators which are sold to households, not so much. All these would stop working if internet providers had a service stoppage the same way Rogers had done, and these effects would have been already felt with the Rogers denial of service.

Too much is riding on the internet, and too much is riding on only a small handful of service providers. So much so, that we appear to take the internet for granted the way we take water, electricity and sewage for granted. We just assume it works and people are doing their jobs.

But the internet is very different from these other public utilities, in that there are too many variables involved in providing people with decent service. Networks can get hacked, and DDOS attacks are common enough to brandish its own acronym. Weather events happen and can cause regions to have to do without service for some time. As we have seen there are many services, and to have the same provider do them all is like putting all your eggs in the one basket.

It must also be added that the Internet wouldn’t exist without government handouts to the telcoms. It was taxpayer’s money that established the main trunk lines for the internet in the 80s and 90s in Canada and the United States. The infrastructure was practically given away to the major telcoms, and we are now seeing an example of what happens when the internet is controlled by too few companies which are largely unregulated and have little public accountability.

There is more than one major provider in Canada – Bell and Cogeco are other big players that come to mind. But we need more than just a few, so that if a DDOS attack happens, the number of people affected will be limited. Outages such as this can slow down the federal government’s attempt to provide all Canadians with universal high-speed internet by 2030.

With information from The CBC, the Toronto Star, and other websites.

Ubuntu on Windows 10

For some years now, Windows and Ubuntu have been coexisting to a degree, if you enable the Linux subsystem on windows and download the Ubuntu for Windows package from the Windows App store.

It makes it possible to muck about with Windows drivers and the Windows kernel from within a UNIX environment. Even make your own drivers that can send direct commands to the Windows kernel and even the TCP/IP stack. So long as you like the command line, there are some pretty cool tools and languages, such as C/C++, python, perl, and many of the other usual suspects at your disposal. It doesn’t really have support for Java, except as a runtime envirnonment. That shouldn’t stop you from installing JDK manually. You can install the one for Linux or the one for MS-Windows. Your choice. Also, there is no support at all for X-Windows.

The /mnt directory is used to house all of the drive letters that are visible to Windows. Here they are mounted as folders each named after their drive letter.

I can’t run MS-Windows commands like Notepad from the shell; but it turns out the Windows paths are not set by default in Ubuntu. Typing /mnt/c/Windows/notepad.exe allowed it to run. In fact, it can run any windows command, if you take the time to fix the $PATH variable. In addition, the Ubuntu subsystem doesn’t yet support the reading of ext3 filesystems, although it has no problem reading NTFS filesystems. An EXT3 driver I tried was able to identify and mount EXT3 filesystems (assigning it a drive letter) from within Windows, but no files were visible. I was offered to format the drive, but I declined. So, I wasn’t sure of the rationale for even having this driver if I can’t see any files.

Apart from that, it appears as if the main architect of the ext2 driver project, Matt Wu, has abandoned the project and has reduced his website to a blank webpage. I don’t see any updates on SourceForge later than 2015.

Cygwin’s (many) downsides

Cygwin is a free (as in freedom) open-source suite which tries to be a POSIX-based subsystem that runs on top of MS-Windows. It tries to behave as if it can do all tasks that Windows can, as if it were a wrapper for Windows. But essentially, even with an X-Window manager, it ends up being just another windowed application with windowed apps running inside it, which can be minimized so you can actually work with MS-Windows itself when you want to.

I wish to say at the outset that this is more of a review than anything. There is a lot of important info missing to construe this discussion as a how-to manual for un-installing or fixing a Cygwin system after a Windows reinstallation. If it were such a manual, this article would have to be much, much longer. In reality, I am really just venting frustration as to how Cygwin, a “program” (for lack of a better word) which I have been using for over a decade, is still very far from getting its act together.

An Eterm, running under WindowMaker. Yes, I was using the EMACS Cheat Sheet as wallpaper!

It has been a hobby of mine to make something of this subsystem for some years, and I have found it most useful as a programming environment. It has as much support for perl, python, C/C++ and vim as you like, and can even run windowed file managers, web browsers (among them, chromium, and lesser known ones like Opera and Midori), and editors like XEmacs. It has wide support of the standard window managers, such as GNOME, KDE, xfce, lxde, fvwm2, enlightenment, WindowMaker, right down to twm. And because all of this runs in a glorified window under MS-Windows, I can switch back and forth to and from MS-Windows whenever it suits me.

If Cygwin doesn’t have the packages for a “free” (as in beer) computer language, I found I can just install a Windows version of it under Cygwin, and that is fine. All Cygwin executables are “exe”, just like Windows, so I can also run Windows commands under a Bash shell. I wanted the latest Java from the Oracle website, and I found I was able to just unpack it somewhere, under, say, /opt, and link its executables to /bin or to any directory defined in my $PATH. Or, of course Java provides a “bin” directory which you can add to your $PATH without the need for making symbolic links.

All tickety-boo if you can get Cygwin up and running. Most applications ported from other unix systems will work if you recompile from source and run the configure script. Others will compile and install after some minor editing.

The downsides of Cygwin are apparent from the point of installation. When you first install Cygwin, the installer is somewhat cryptic, although you might be able to figure most of it out. The installer allows you to decide what packages you want and which ones you don’t. But it is really an illusion. If you want stuff such as your window managers to work on Cygwin, including your chosen X-Window manager, then just install everything. Maybe decide which window manager (or managers) you want and which ones you don’t want. I also was picky about the texmf language packs, which slow down the install to over 6 hours, so I do take the trouble to deselect most texmf language packs that are not English or which don’t use the “Latin” alphabet, while choosing any math or other academic fonts. Being otherwise indiscriminate about package selection means you have to live with scores (or possibly hundreds) of programs and window managers you will never care to use. My installation is typically about 26 gigs unpacked, spanning over 1 million files (1,017,806 files, to be exact) in some 65,000 folders. That is not counting my /home folder.

Another thing to know is that Cygwin has no uninstall tool to uninstall itself. So un-installation of Cygwin is infinitely more difficult than installation, for reasons we shall see.

I said earlier I found the installer cryptic. What I mean is that the installer has to download and install each package one at a time, which is a bugger if a remote server goes down or hangs. And, especially with those texmf language packs it appears to hang, when in reality it is just plain being slow. If you stop the installer and start it again, you get no indication of whether it remembered where it left off. What you do is behave as if it does remember, and click install. It does pick up from where it left off, but it is not very reassuring about it.

And God help you if you, for some reason need to reinstall MS-Windows. This is where you find when you go back to the directory with the installation, that it has introduced its own permissions, but that is not the worst permission problem. The ownership of the distribution becomes hard to untangle, in large part because when you reinstall MS-Windows the user and admin accounts you created become reduced to SID numbers of users no longer known to the system. And of course, you can remove those unknown users with Windows’ Properties, and reassert your ownership similarly, or by using the “takeown” and “icacls” commands which their cmd shell provides (running as Administrator). This takes hours when the number of files is over 1 million with 65,000 folders. This is slowed down further by the fact that there are several files and folders which have un-knowable permissions and un-knowable ownership which require you to change tactics when that happens. After an evening and the following morning, I was able to get rid of the now-bogus users while respecting other owners as much as possible. Using the inheritance option in MS-Windows has to be done judiciously, respecting that different folders have differing sets of system permissions. Some have no system permissions (just user and group permissions), while others have strange ones like “Creator Owner”, “Creator Group”, “None”, and “NULL SID”.

Some examples of strange “users”. “Everyone” isn’t considered strange, and includes all users on your system.

If you are able to untangle the Cygwin permission problems after re-installing MS-Windows, then congratulations! You are now at a point where you can decide two things: 1) you can still configure an icon to run a shell under mintty and content yourself with a bash shell as a reward for your work on permission changing; or 2) you can delete the entire Cygwin directory tree and decide if you want to reinstall again. Both prospects are hard-won, and here you are. I wish to emphasize that option 2 is not a joke. Deletion would have been impossible without taking ownership and fixing the permissions. It just sounds like a joke.

Notice that there is no choice “3” for trying to run X-Windows or a window manager. X-Windows will complain one way or another about not being able to find :0 (the root window), or will give an X window briefly, which crashes in seconds with no error message logged. Some X apps work, such as the aforementioned mintty, but except for shell commands, that’s it. If you wanted to run an application that needs any of the X-windows widgets, then you have to delete the whole thing (except possibly /home) and install from scratch. In other words you are basically screwed in all but the bleakest of ways if you reinstalled MS-Windows.

Over the years there had been several reasons for reinstalling. Sometimes it was to freshen a windows installation which was becoming increasingly sluggish and full of problems. “Freshening” a windows installation involves, for me, a formatting of C: drive. This is not so bad for me. Only programs and system files go on my C: drive. My documents and other files are on other physical hard drives. My Cygwin installation is also situated on one of the other physical drives, so it doesn’t take up valuable room on C:. So, I don’t feel as nervous about reinstallation as some would; but there is that darned Cygwin distro I have to reckon with sooner or later. You are screwed if you sort out the permissions under Cygwin, and screwed even more if you don’t.

This is /etc/fstab in its entirety. The edited line on the bottom contains the “noacl” option needed to fix this file permission bug.

As a post script, I found out how you get NULL SID, as well as incorrect ordering of permissions on many of the Cygwin files, the source of the majority of my permission headaches. The /etc/fstab has just one uncommented line, a filesystem called “none” allowing all of your Windows drives to be herded under /Cygdrive. This is supposed to have the advantage of allowing you to navigate to any physical drive or partition on your computer entirely within Cygwin (which is what it does). A missing option needs to be added: “noacl” (quotes omitted). This prevents Windows from trying to assign a user SID as if “none” was a user, thereby fixing many of the permission headaches.

I don’t understand why the designers of Cygwin don’t add “noacl” before they distribute it. I think the majority of us are running some form of NT-based windows system: Windows 2000, XP, 7, 8, 10, and now 11 are packaged with most computers these days, and their hard drives are usually NTFS. These bugs are specific to NTFS systems, and these bugs don’t show up on FAT-32 filesystems, which don’t store info on ACLs, SIDs, or anything of the sort.

The Cygwin website discusses these issues, but it seems that Cygwin is trying to be POSIX compliant when Windows obviously isn’t trying to be. If they are choosing MS-Windows as the host system, they will have to do things their way and not try to fight it with Cygwin’s “correct” way, or to disenfranchise the majority of their users for the sake of backwards compatability. Would it kill the owners of FAT-32 filesystems, whom I think are in the minority, to delete “noacl” for the sake of the majority? Once the system is installed it is too late to do it then, since by then the installed apps will all have the permission bug.

In the event you decide you wish to delete the whole shebang after hours of sorting out permissions, there is one little tiny file that completely thwarts nearly all attempts at deletion. I have found it on two of my installations, and it was a problem in that specific file. It is the file located at /usr/share/avogadro/crystals/zeolites/CON.cif, relative to the Cygwin top-level folder. It cannot be deleted, and its permissions and ownership cannot be changed or even known to humans. The reason Windows appears to go braindead with this file, is because of the filename. CON is a reserved word in MS-Windows, short for “console”, going back to the days of MS-DOS. So is naming your file LPT1, short for “line printer”, a Windows reserved word with the same MS-DOS heritage. You can’t delete it with anything in Windows, so you need a POSIX tool, like, ahem, Cygwin, to affect the deletion.

So I deleted CON.cif using my later installation of Cygwin, and I was thus able to delete the entire directory tree as a result. More to this issue is: what happens when you need to delete CON.cif and have no intention of reinstalling Cygwin? Stack Exchange has a whole discussion on this which makes my long story even longer, so I will end my article here.

Shopping for a printer, and network troubleshooting mysteries

Brother HL-3170CDW. My current printer with the dodgy network card.

I vaguely remember the network card not working when I bought the new Brother computer I am currently using. But I decided about a year or two later to test what is wrong using my network tester, which can test a connection over a cat 5 cable. Plugging both ends of the cable into the tester seemed to work. So I know the cable is fine. Plugging one end into the printer and the other end into the tester, it seemed to work also. Then testing the connection on the router using the same cable worked as well, in the sense that the lights on my testing device were blinking in the right order. But when I finally hooked up the printer to the router, I noticed no lights were flickering on the router end. So, maybe that port was the problem? I took a working connection from a switch I didn’t care about, and plugged it into the same ethernet port on the router. Suddenly that port came to life. So, I finally recalled that the printer had something wrong with the network card. I had the card on the printer replaced early on, and it was defective again. That, I remember, was the reason why I hooked up my printer by USB. It has wireless capability, but 1) both my desktop PCs have only cat-5; and 2) the printer is not visible on my router’s wireless settings anyway.

I stuck with the printer because it was a simple printer that did duplex printing and did it well. But I am going to have to consider shelving this printer and getting another one which with a network port that works. So far, my best bet is to order online, since almost nothing appears to be in stock. I am reading Amazon reviews of some decent printers and there are many harrowing tales about multiple DOA printers (original was DOA and so was its replacement), sketchy tech support – and this is for a Xerox printer, which one would have thought would be an established brand. This will take some thought, and maybe a bigger budget than planned. And this is for a single-function duplex laser printer. I am not asking for anything crazy here.

For now, I refreshed the drivers, and cleaned out the system using CCleaner. The printer will have to serve me for now.

I was reviewing my computer’s specs using a software called “Speccy”. You might have heard of it. It is one of those small bits of software that came with my “Pro” copy of CCleaner. It just tells you details about your system and components. I wanted a printout of what I saw on the screen regarding what drive letters are intended for which empty ports on my card reader. And about 30 pages of printing later, … not much on that, but a lot on each and every other detail I wasn’t interested in. It would have still kept printing had I not hit the “cancel” button on the printer. I specified double-sided printing, but because I changed the driver to the functioning printer in front of me, the duplex setting was forgotten about by the driver, and the 30 or so pages are all single-sided. What a waste.

Looking at the website for Consumer Reports, I notice that the field is dominated by Brother or HP printers. Xerox is way down below the top 10, receiving scores in the 50s, but no single-function printer I looked at scored above 68. Also, the most expensive are not necessarily the highest-rated. Another workhorse brand, Lexmark, also is not in the top 10.

Brother HL-3230CDW. Not all that different.

My top two criteria are “Predicted Reliability” and “Ink/Maintenance”. The latter criteria, I found out later, are mostly relevant to inkjets, since it refers to the ink used during their self-cleaning/maintenance cycle. All printers shown to me were duplexing color lasers. No printer scored 5/5 for reliability. The top-rated printer, a Brother HL-3230CDW (CR score of 67), has the familiar problem in that it refuses to print in black (or print at all) if a color cartridge runs out. My printer is similar in this regard. I like that it duplexes, something I am willing to pay a little extra for. But apparently it does not print color photos very well, a “feature” shared by these sorts of printers, apparently. The tray takes up to half a ream. Prices vary from $350 to over $550 depending on the seller.

Just below that is a Hewlett-Packard Color LaserJet Pro M255dw, given a rating of 66. Photo quality is still bad; and this one will reject generic cartridges, forcing you to have to shuck out the full price of HP cartridges. Horrible business practice. The printer I have right now is using generic cartridges. In fact, it has never used cartridges from the manufacturer since I had purchased it three years ago. The four generic cartridges cost me a total of $200. Each HP cartridge costs around $80, and four of them would pay for the cost of the HP printer they fit in. CR estimates that with the long life of the cartridges, your cartridge outlay will be in the neighbourhood of $500 over 4 years. I am converting to Canadian dollars and lowballing. Lexmark printers also insist on their proprietary cartridges. Xerox doesn’t appear to suffer from this problem; and Brother certainly doesn’t. CR’s priorities are not the same as mine, it appears.

The ultimate HP AIO color laser printer. The fastest, and boasting the lowest printing cost per page. At a $10K price tag, it could only be the lowest cost per page for me if they were giving away free reams of paper. It might pay off in a business setting where the printer is in very high usage, due to its monthly duty cycle of 125,000 pages.

Generally, the ratings are depressingly low. Higher ratings (starting at 78 and going down) are given to monochrome AIO (“all-in-one”) printers, and I am not interested in that. Many of them are also mostly suitable for offices. When it comes to color AIO models, the scores drop to 70 starting with a Brother MFC printer. These generally have higher price tags. AIOs generally print, scan, fax, and photocopy. I haven’t had the need to photocopy, and have separate scanners which perform much better than the AIOs I’ve used. And is faxing still a thing with anyone? Most of us just receive documents sent as emailed attachments these days. In my experience, the fax feature, when I had bothered to hook it up, is only just an avenue for a grifter to send junk faxes to me, at the expense of my ink and my paper.

I guess while CR wouldn’t have known that I have a high-performing scanner and have no need of an AIO, it has helped me to clarify my own priorities: 1) Color laser; 2) duplexing (inkjets do color better, but I am not sold on inkjets doing that on both sides of the same sheet without bleeding, and lasers have a lower per-page cost); 3) reliability with a cat-5 (wireless is OK too, but only if cat-5 is working); 4) compatability with refillable, aftermarket toner cartridges.

As a postscript, my outlay for my working, networked printer was $0.00. I turns out that there was nothing wrong with the network card. What tipped me off was that I got the printer to print out the network information, and noticed the printer had an IP address, and knew the IP of the router. Paradoxically, the printer was not visible anywhere on the network. So, rather than untangle that, a factory reset was all that was needed.

My Introduction to the Raspberry Pi

I was recently sent a Raspberry Pi 4B computer, disassembled, with little introduction or fanfare. I have heard much about the miniature hobby computer over the past decade, but hadn’t actually used one or seen one up close until now. I put it together, and observed that it didn’t POST on its own. Being a tiny motherboard, it has no BIOS at all, and is completely dependent on the installed operating system for hardware detection and testing. I downloaded and installed a Debian image made for Raspberry Pi using their imager you could download from their website.

It boasts a quad-core ARM Processor (Revision 3). It is not your average Intel processor. All of the largest chips on the motherboard each take up no more than 1 square cm, and the heat sinks are so small that they attach like little aluminum decals to their processors. It comes with 4GB RAM installed, and a place for a Micro-SD for the operating system and limited storage. It can support up to two 4K HD monitors at 60 frames per second, each connected with micro-HDMI connectors. It has connectors for legacy Type-A USB2 and USB3 devices. I am using the two USB2 connectors for the mouse and keyboard; and the other two for “anything else”. There is a third Type-C USB3.1 connector which is used for power. The power switch itself is on the same cable as for the power supply. You can also hook up a standard Ethernet cable for connecting to a network. If you don’t have ethernet, the Pi boasts a WiFi receiver. The motherboard goes inside a plastic system box about the size of a small pack of cigarettes.

On the motherboard are pinouts which resemble the ones used for the old parallel ATA connectors from decades ago. After learning how to send power to which pin through a comnputer program, the possibilities of operating small electrical devices (lights, speakers, an arduino device, what else …?) expand greatly.

The installation had a few hitches, since it appears I cracked the first two micro SD cards either on the supplied USB adaptor, or on the micro-SD mount provided underneath the motherboard. I went and purchased a third, a 32GB Sandisk Ultra (I could have gone as low as 4GB, but this was the smallest sold by Staples). I installed the OS using their installer again, and this one booted. Since I was at home, I didn’t have access to my supplied monitor, keyboard and mouse, so I just reconnected existing devices at home, and was rewarded by a rather nice X-Windows display.

The CanaKit Raspberry Pi 4: A $100 Office PC and More ... — The default desktop for Raspberry Pi 4, with some icons added. It comes with a minimal set of default apps, and access to a BASH shell.

The Pi is bluetooth-enabled, but I hadn’t tried it yet. I did plug in a wireless adapter for a Logitech keyboard/mousepad combo (the K400 Plus), and it worked instantly.

Now, since I was given this computer as a teaching tool for students, I made the default account password-protected, and created a new account named “student” with fewer priveleges: they can’t install or remove software, and are not part of the “games” group. The first restriction appears effective when I tested it, but the latter exclusion from the “games” group still meant they were able to play some games, for example those that originate from the python graphics libraries. My students, like teenagers everywhere, have at least a mild addiction to computer games, so I decided to remove them as a possible distraction. Since all students will use a single account, they will need their own USB thumb drive, since there won’t be many other ways to save files edited on the Raspberry Pi. However, since it has WiFi, there is always G-Drive or OneDrive.