[PTTLink] Community FAQ - January 2021

[PTTLink Admin]

The PTTLink Team understands there are a number of unanswered questions 
from the community. The Team would prefer focusing on open source work, 
and our attempts to get community involvement back into the network, 
rather than addressing these “finger-pointing” style questions. However, 
we also recognize the community deserves answers to those questions. 
Please understand that some questions  can only be addressed by publicly 
holding those responsible accountable for their actions. Our answers are a 
Team effort and took considerable time to compile based upon everyone’s 
understanding of what led to this. As such, we hope this answers the 
outstanding questions, and allows everyone to move forward with their 
community efforts.

1.  Were there any SLAs or Agreements associated with the hosting of 
services for AllStarLink, Inc. or the AllStarLink Network?

Systems on AllStarLink before the board’s changes on 29 December 2020 were 
run much as you’d expect from any radio club.  Systems were volunteered by 
the team and community because they were needed. There was no agreement 
between the system owners and AllStarLink regarding how things were run, 
what to do with data, etc.  As resources were needed a call went out to 
the community and the community graciously answered.

It was left to the Admin Committee to identify the best use for resources, 
configurations, and maintenance. This charter of the Admin Committee was 
codified in the wiki and approved by the Board of Directors from day one 
of AllStarLink, Inc.  You can find the Admin Committee charter here: 
https://wiki.allstarlink.org/wiki/Admin_Committee

The Admin Committee identified what was needed for virtual machine (VM) 
specifications, etc.  All systems in use were run within a virtual 
environment, and at no time did the AllStarLink Board of Directors have 
physical access to any of the servers that ran the network. Services were 
mirrored in multiple locations to spread the load throughout the 
volunteered systems and to reduce the chances of creating a single point 
of failure. This resulted in a design that at any point in time would 
allow for load-shift to allow for issue isolation and scalability.

2.  Regarding the removal of slack access and removal of node numbers.

Public examples of these actions can be found at the following links:

- Slack account removal is documented in Stacy Olivas’ (KG7QIN) reply to 
the board’s letter here 
http://lists.keekles.org/pipermail/app_rpt-users/2020-December/020954.html

and this post 
https://community.allstarlink.org/t/slack-trouble-today/18019

- Missing nodes were removed from multiple Admin Committee member’s 
accounts. Some have been returned, however many low number nodes have yet 
to be returned.  You can find examples of this documented here 
http://lists.keekles.org/pipermail/app_rpt-users/2021-January/021009.html

- Finally, the Letter to the Community of Jan 3, 2021, as it appears on 
the community site, has been edited. We want to make it clear you should 
*ADD* the PTTLink register lines, not replace the AllStarLink registration 
lines.  The entire goal of adding the lines is to allow registration on 
both systems, we do NOT want to replace the AllStarLink network but rather 
provide additional well managed and maintained registration servers and 
backend resources. You can find the Letter to the Community here (which 
includes links to both the post on Reddit and the edited copy on the 
AllStarLink Community forum): 
http://lists.keekles.org/pipermail/app_rpt-users/2021-January/021015.html

3.  Regarding Opt-In, our focus on community building, and moderation.

Please see our Community Update dated Jan 7, 2021 which is located at 
https://www.reddit.com/r/pttlink/comments/ksupw6/pttlink_community_update_7_jan_2021/

4.  Network design.

The PTTLink network is the original AllStarLink network which consists of 
primary and secondary locations spread throughout the world including the 
United States, Canada, Europe, and Asia. At each primary location there 
consists a registration server, clustered database server, and some 
miscellaneous ancillary services. This allowed the Admin Committee to not 
only ensure app_rpt clients could register should any one site go down, 
but also ensure no data loss in the case of an emergency or catastrophic 
failure. Secondary locations only maintained a registration server to help 
offset load.

The registration servers were previously modified versions of Asterisk, 
which added additional stability.  However, these were in the process of 
being migrated to new software (more details on this are available in the 
next section). Upon successful registration, a database table is updated 
with the registration details, which in turn causes a MySQL trigger to 
create or update the DNS records served by PowerDNS.  This provides the 
core DNS lookup services documented at: 
https://wiki.pttlink.org/wiki/DNS_Servers and the code for the MySQL 
trigger is available at:  https://github.com/Apprpt-Central/dns-lookup

Every site was, and continues to be interconnected with secure VPN 
connections.  Host based firewall rules ensure controlled access to 
systems were only permitted via either specific IP addresses or via a 
bastion host acting as a proxy. All access was and continues to be logged. 
Status of the network was made public by a dedicated dashboard that shows 
the status of sites, as well as registration counts for each server.

This design adds complexity to support a very large network, however, our 
mandate is to keep creating a network as easy as possible, and as such, we 
pledge to design software and help networks of all sizes.

5.  Accusations of not being supportive of the board.

It has been commented that the Admin Committee did not support design 
suggestions from the board. This is quite the contrary, the Admin 
Committee took suggestions from anyone that provided them, and weighed 
them equally to best serve the network and community.

One such example lies in the new registration software we believe to be 
currently in use by AllStarLink.  This software was originally brought 
forth to the Admin Committee by Rob Vella (KK9ROB). The concept was 
approved and embraced by the Admin Committee who also identified and fixed 
multiple bugs while Rob appeared to have become unresponsive. By the time 
of the cutover, over half of the registration systems were running this 
code.  Several improvements have been identified and made to increase the 
stability of this code since its initial deployment.

After AllStarLink, Inc had received non-profit status, it was in receipt 
of Amazon Web Services (AWS) credits which it had applied for and 
subsequently led to community.allstarlink.org being relocated there by an 
AllStarLink Board Member. On several occasions, the Admin Committee 
requested access to the AWS infrastructure so we could make use of and 
take advantage of these resources; we were told multiple times this would 
not be possible as there was no funding available. This was not the case 
as they have moved all services other than Community to the Google Cloud 
Platform which does incur a recurring cost.

Finally, the Admin Committee recognized the lack of communication between 
our team and the AllStarLink, Inc Board of Directors. Multiple attempts to 
raise board members on internal communication channels would go weeks 
without an answer, if any was given at all. Requests for representation at 
board meetings to help bridge this gap were ignored. While a 3rd party did 
volunteer for this role, the lack of commitment and follow-through by the 
board resulted in little more than the continued empty assurances and 
broken promises at transparency and communication which had been promised 
to the Admin Committee many times.

Why was there so much silence/inactivity?

Even though the board did not hold or post minutes or give any 
communication or direction, the admin team continued to maintain the 
Allstarlink Systems and work on new code for the registration servers, 
development of the app_rpt and other associated software which supported 
the Allstarlink System.

The Admin Committee dedicated multiple years, countless hours of sleepless 
nights, and made every reasonable effort within their power as volunteers 
to keep the AllStarLink network running for the enjoyment of all hams. In 
the end, this team is a group of hobbyists that feel that they were let 
down by the AllStarLink, Inc. Board of Directors for their lack of 
communication, failure to understand and unwillingness to explore the 
reasoning and technical constraints behind the design of the AllStarLink 
infrastructure. Resulting, more often than not in service outages caused 
by the implementation of their ideas without involving the Admin 
Committee, and placing it in production without adequate testing. During 
the numerous attempts at having meaningful discussion with the board 
regarding redesign, the reply often came back “The board is technical too” 
in an attempt to justify their actions.

6.  Asterisk GitHub Repository.

The “AllStarLink” GitHub repository started as a single account with the 
email address of developers at allstarlink.org.  GitHub at the time did not 
offer much to free accounts, so this was very much just a public place to 
stash the code that wasn’t on the old www.allstarlink.org site/server. 
Stacy was given the user account credentials (several people had them) and 
fixed some things up with regards to inviting others as repo 
admins/maintainers, etc.  Unfortunately, one of the maintainers pushed an 
unreviewed pull request directly into the master branch without telling 
anyone.  Fortunately, Stacy saw the notice sent to the developers@ email 
(which was an alias for several people at this point) and was able to 
login and back the commit out, and push it into its own branch where it 
remains to this day.  To ensure that this wouldn’t happen again, the 
developer account was removed and instead made into a personal 
organization with the name of AllStarLink.  Stacy then went through and 
set up branch protections that require two people to review code submitted 
to develop and main, among other fixups and integrations in an effort to 
prevent this from happening again.  The AllStarLink GitHub repository has 
always been a separate entity controlled by the developers, of which Stacy 
was originally brought on by Bryan Fields (W9CR) as the lead developer for 
AllStarLink and later also became an admin.  Owners of this new 
organization included Tim Sawyer (WD6AWP), an AllStarLink, Inc. board 
member, who was responsible for the unreviewed commit which led to this 
new personal organization being created.

The size and number of people working on AllStarLink’s infrastructure 
needed a central repository to hold configuration files, internally 
used/developed software like the stats server and web portal, etc.  A 
private Gitlab repo was set up to contain this as well as the continuing 
work that Stacy had been making on his Asterisk 1.8 port of app_rpt.  This 
was done due to the climate regarding hamvoip claiming code as their own 
and not releasing any changes per the terms of the GPL.  Stacy was able to 
update the 1.8 port with the 0.324 changes to app_rpt and pushed the 
version number of 0.399.  Additionally, he also started work on fixing 
some of the known issues with various channel drivers.  Unfortunately, the 
ever-increasing time being the main admin of the AllStarLink 
infrastructure as well as severe audio issues from URIs featuring the 
CM119B reduced Stacy’s available time to work on core development.

At the time of the board’s decision to remove the Admin Committee from all 
services, a small group had again formed to begin moving the app_rpt 
project forward with enhancements to the registration and statistics code, 
as well as multiple bug fixes including URI EEPROM handling, just to name 
a few. The development team was never allowed access to the AllStarLink 
slack to discuss the updates and a temporary home was donated to drive 
this effort.

The app_rpt and related code is licensed under the GNU GPL and is 
dependent upon and a derivative of Digium’s clearly marked and licensed 
Asterisk 1.4.23pre code base, itself licensed under the GNU GPL.  Due to 
the innumerable and often uncredited contributions to this code that 
extend, complement or enhance app_rpt, which in turn are also licensed 
under the GNU GPL, it is the opinion of PTTLink that the repository 
ideally should remain a separate entity from any respective network.  The 
app_rpt application and related code is an open-source resource that is 
for all to use and benefit from.  However, responsible stewardship of this 
repository is a must to ensure that it remains a viable asset. The 
development team agreed that PTTLink should retain stewardship of this 
repository moving forward. We do however look forward to all community 
members raising issues, and submitting pull requests against the codebase 
to further the community use of the system.

Additionally, you can find information regarding the updated Asterisk 1.8 
port code recently moved from Gitlab to Github and renamed ASL-AsteriskNG 
at https://wiki.pttlink.org/ASL-AsteriskNG.  This will be the next 
generation of app_rpt client and is slated for the ASL 1.10 release.

7.  Allocation of Resources from the American Registry for Internet 
Numbers (ARIN).

The claims made by the AllStarLink, Inc. Board of Directors and their 
supporters regarding the ownership of ARIN resources by Jim Dixon (WB6NIL) 
(SK) on several forums are false.  Jim did not own these resources, he was 
merely a technical contact.  The majority of these resources go back to 
the days of Dr. John Postel, and were held under various companies, most 
of which may or may not be associated with Jim.  There’s roughly 12 /24’s 
of IPv4 address space and a couple AS numbers.  The ARIN registry 
associated with these resources do not contain any admin contacts for the 
organizations listed.

Bryan was asked to become a steward of these by the AllStarLink, Inc. 
Board of Directors  prior to Todd Lesser (KM6RPT) joining.  At that time 
Bryan was asked to investigate the options they had as there were possible 
finances available with the sale of the IPV4 address space.  The board 
requested Bryan to add Mike Wolthius (KB8ZGL) and Stacy to be the admin 
contacts, but as Bryan only had tech access privileges, they could only be 
added as tech contacts per ARIN policy and their access constraints 
imposed on Bryan.  Once completed, Bryan was told AllStarLink, Inc. was 
working with Jim’s estate to get these ARIN resources assigned to them. 
During this process, the AllStarLink, Inc. Board of Directors were told 
that any attempts to update the admin contacts needed to include the proof 
that Jim was the owner of the various companies and that ARIN would need 
to see any agreement so as to approve it.

After Bryan was informed this agreement was in place, the AllStarLink, 
Inc. Board of Directors refused to share this. Bryan asked formally for 
the Assignment Agreement to ensure he was assigning what he held as a tech 
contact in trust properly. The repeated refusals caused Bryan to doubt the 
Agreement contained all requirements necessary to prove to ARIN of its 
legitimacy to transfer ownership of the resources.  Bryan contacted ARIN 
and it was determined the best course of action was to remove all tech 
contacts from the resources, which was promptly done.

Finally, after many months of research, Bryan tracked down the Agreement 
from Jim’s estate as it had been submitted to the Federal Communications 
Commission  (FCC) in an application requesting the transfer of the WB6NIL 
callsign.  This action made it a matter of public record in the Universal 
Licensing System (ULS) for the WB6NIL reassignment to AllStarLink, Inc. 
(who had previously applied for a callsign as a radio club).  While the 
Agreement does list several items in its Appendix A related to the ARIN 
resources, and defines it as assigned property, the Agreement not only 
does not identify Jim to have been the owner of the items and/or 
associated businesses, it does not actually assign any of the items in 
Appendix A to AllStarLink, Inc.. Additionally, the Agreement does not have 
a valid signature from the AllStarLink, Inc. Secretary on it.  Both are 
major oversights in the Agreement resulting in the ARIN helpdesk making 
the determination that the Agreement is inadequate evidence of ownership 
and denied any transfer of resources to AllStarLink, Inc.  Bryan informed 
AllStarLink, Inc that he was unable to update ARIN’s records to transfer 
control of the assets as AllStarLink, Inc. did not have a valid Agreement 
to recover admin access to the resources.  AllStarLInk, Inc. was also 
informed of the ARIN procedures required to correct this when Bryan stated 
he didn’t feel comfortable to continue.  Todd promptly threatened to sue 
Bryan for taking these actions on behalf and at the request of the 
AllStarLink, Inc. Board of Directors.  You can find the document in 
question here listed under Attachments as “Agreement from Heir”.  Refer to 
section 6 for details: 
https://wireless2.fcc.gov/UlsApp/ApplicationSearch/applAdmin.jsp?applID=11845728

This was all done in an attempt to not only protect the IP and ASN ranges, 
but also the AllStarLink, Inc from the potentially fraudulent actions of a 
Board member. Additionally, there is also a real possibility that the IPs 
would be reclaimed by ARIN as the organizations that do own the IP address 
spaces could be found in default and no longer operational. There can be 
no financial remuneration to anyone involved including Bryan, Jim’s 
estate, or AllStarLink, Inc. until ARIN has enough evidence to modify the 
admin contacts.

8. A few last words.

Again, the PTTLink statements and letters are the product of a team 
effort, and not just that of a single person. Every team member has the 
opportunity to review and hold the others on the team accountable for what 
they write. We strive to be transparent, forthcoming, and always stating 
the truth, even if it is uncomfortable.

PTTLink is not against AllStarLink but rather a group created as a 
consequence of the AllStarLink, Inc. Board of Directors actions.  PTTLink 
exists to maintain, and enhance, the registration and associated ancillary 
services system for the benefit of all.  One of the core values of PTTLink 
is to promote, stimulate and provide enhancements and growth to the 
amateur radio community at-large.  To this very end PTTLink has already 
committed to, and begun, working on an agnostic multi-network capable 
version of app_rpt and associated programs. This includes, but is not 
limited to, open-sourcing key components used to create your own network 
for registration and DNS lookup, the establishment of the network prefix 
schema and associated repository for multi-network support.

It is the desire of the PTTLink team to have this be the final letter 
addressing this matter.  The team wishes to move forward and work together 
with the amateur community at-large to support, innovate, and grow this 
amazing community and hobby.

Signed,
The PTTLink Team