NNTP - how to check for removed newsgroups ?

[R.Wieser]

Hello all,

Simple question : using the NNTP protocol - How do I check for newsgroups
that have been removed ?

I don't really want to go thru the list of all newsgroups I currently have
stored and check, one by one, if I they are still present on the newsgroup
server.

Remark: I found the command to find all *new* newsgroups (since a certain
date), but have not been able to find its opposite.

Regards,
Rudy Wieser

[Auric__]

Post by R.Wieser
Simple question : using the NNTP protocol - How do I check for newsgroups
that have been removed ?
I don't really want to go thru the list of all newsgroups I currently have
stored and check, one by one, if I they are still present on the newsgroup
server.
Remark: I found the command to find all *new* newsgroups (since a certain
date), but have not been able to find its opposite.

There isn't a method defined in any RFC to do so, because there are a number
of hurdles that make it infeasible:

- *WHO* decides what groups to include on that list?
- How far back do you want to go/removed since *WHEN*?
- For each user, the server would need to remember the last time that user
checked the groups, and what groups have been removed since then. *WHERE*
does the server store this information?
- *WHY* would the server retain information it has already removed?

The *only* method to do this is using the "LIST" command.

Alternately, you could send the "CAPABILITIES" command and see what the
server returns. Maybe your chosen server has a method to do what you want.
(Don't count on it.)

[Auric__]

Post by Auric__

Post by R.Wieser
Simple question : using the NNTP protocol - How do I check for
newsgroups that have been removed ?
I don't really want to go thru the list of all newsgroups I currently
have stored and check, one by one, if I they are still present on the
newsgroup server.
Remark: I found the command to find all *new* newsgroups (since a
certain date), but have not been able to find its opposite.

There isn't a method defined in any RFC to do so, because there are a
- *WHO* decides what groups to include on that list?
- How far back do you want to go/removed since *WHEN*?
- For each user, the server would need to remember the last time that user
checked the groups, and what groups have been removed since then.
*WHERE* does the server store this information?
- *WHY* would the server retain information it has already removed?
The *only* method to do this is using the "LIST" command.

...and then comparing the downloaded list to the local copy.

Post by Auric__
Alternately, you could send the "CAPABILITIES" command and see what the
server returns. Maybe your chosen server has a method to do what you
want. (Don't count on it.)

[R.Wieser]

Auric__,

Post by Auric__

Post by R.Wieser
Simple question : using the NNTP protocol - How do I check for
newsgroups that have been removed ?

There isn't a method defined in any RFC to do so,

I didn't see one either. Doesn't mean I could not have overlooked the
specific RFC and the appropriate command in it. Hence my question.
"infeasible" ? I beg to disagree.

Post by Auric__
- *WHO* decides what groups to include on that list?

Who decides which groups to include in the "new newsgroups" list?

Post by Auric__
- How far back do you want to go/removed since *WHEN*?

Thats the same problem as for the "new newsgroups" list. Maybe use the same
method ?

Post by Auric__
- For each user, the server would need to remember the last time that user
checked the groups, and what groups have been removed since then. *WHERE*
does the server store this information?

As the "new newsgroups" list doesn't suffer any such problems I do not see
why an "old newsgroups" should suffer them.

Post by Auric__
- *WHY* would the server retain information it has already removed?

For the same reason it retains information about which "new" newsgroups it
has ?

Post by Auric__
The *only* method to do this is using the "LIST" command.

Nope. I already mentioned another one in the initial post.

But now you mention it, that LIST command can also be used to find any new
newsgroups. But they still implemented a seperate command for it. Why ?

Post by Auric__
Alternately, you could send the "CAPABILITIES" command and see what
the server returns. Maybe your chosen server has a method to do what
you want. (Don't count on it.)

:-) "Don't count on it" :-)

Regards,
Rudy Wieser

[Char Jackson]

Post by Auric__

Post by R.Wieser
Simple question : using the NNTP protocol - How do I check for newsgroups
that have been removed ?
I don't really want to go thru the list of all newsgroups I currently have
stored and check, one by one, if I they are still present on the newsgroup
server.
Remark: I found the command to find all *new* newsgroups (since a certain
date), but have not been able to find its opposite.

There isn't a method defined in any RFC to do so, because there are a number

Rather than infeasible, it's more likely to be trivial. I don't know of a client
that offers the capability, but that's probably because no one has asked for it.

Post by Auric__
- *WHO* decides what groups to include on that list?

The user who sits behind the news client or script that provides the capability.

Post by Auric__
- How far back do you want to go/removed since *WHEN*?

Since the last time this user performed this check against this server, using
this client/script.

Post by Auric__
- For each user, the server would need to remember the last time that user
checked the groups, and what groups have been removed since then. *WHERE*
does the server store this information?

The server would store nothing. The client or local script would store
everything.

Post by Auric__
- *WHY* would the server retain information it has already removed?

N/A, since the server would need to store nothing in this regard.

Older versions of Forte Agent, at least up through 2.0 since that's where most
of my experience was, use a newsrc file to keep track of each group carried by
the designated server. Agent 2.0 has a menu command to "Get new groups", and it
has an in-built command to see a list of only the new groups. New, in this case,
simply means groups that have been added since the last time this person checked
for new groups. Behind the scenes, the command probably builds a new newsrc
file, since that already contains a list of all of the groups, then it could
compare the new file to the old file, diff-like, to see what's been added. That
much is trivial, and once it does the diff, I don't see why there couldn't have
been an additional view that shows the groups that have been removed since the
last check.

It seems to me that any client that uses a newsrc file could be used as the
basis for determining which groups are New or Old since the last time the
command was run. Yes, the newsrc file contains additional information,
especially for subscribed groups, but it sounds like all of that can be ignored
if only deleted groups need to be listed.

[R.Wieser]

Char,

Post by Char Jackson

Post by Auric__

Post by R.Wieser
Simple question : using the NNTP protocol - How do I check for
newsgroups that have been removed ?

There isn't a method defined in any RFC to do so, because there are

Rather than infeasible, it's more likely to be trivial.

Indeed. Proof on point: the "new newsgroups" command.

Post by Char Jackson

Post by Auric__
- *WHO* decides what groups to include on that list?

The user who sits behind the news client or script that provides the capability.

:-) The question could be as easily have been about the user as about the
newsgroup server. As such my answer was deliberatily as vague, to covered
both.

But, if its about the user than I, again, refer to the "new newsgroups"
command, which seems to have solved that problem quite nicely.

Post by Char Jackson

Post by Auric__
- How far back do you want to go/removed since *WHEN*?

Since the last time this user performed this check against this server,
using this client/script.

Simpler answer : since a user-provided date-time. Like the "new newsgroups"
command already does.

Post by Char Jackson

Post by Auric__
- For each user, the server would need to remember the last time that user
checked the groups, and what groups have been removed since then. *WHERE*
does the server store this information?

The server would store nothing. The client or local script would store
everything.

Auric is thinking of a statefull environment where none is required.
Over-complicating stuff is an occupational hazard to a programmer. :-)

Post by Char Jackson
then it could compare the new file to the old file, diff-like, to see
what's been added. That much is trivial, and once it does the diff, I
don't see why there couldn't have been an additional view that shows
the groups that have been removed since the last check.

My newsgroup reader just flags the new newsgroups in its local list, meaning
that I can ask for them (long) after they have been added. Though I have no
idea what the retaining time (if it works that way) for that "new newsgroup"
flag is.

Post by Char Jackson
It seems to me that any client that uses a newsrc file could be used as
the basis for determining which groups are New or Old since the last
time the command was run.

Aurics suggested to use the LIST commands result for that.

But although that will work, its also rather wastefull to have to download
all the newsgroups (for me "only" 27,000+ of them - for just one newsgroup
server) just to be able to do a diff. Hence the "new newsgroups" command
was added. Instead of retrieving thousands of groups (normally) only a
handfull (if that much) will be returned.

As a test in which I asked for all new newsgroups since January 1, 2023
(read: a one-and-a-half years worth) I got just 40 of them.

As I recognised the wisdom of that "new newsgroups" command I imagined that
an "old newsgroups" command could exist for the same reason.

But alas, being able to easily purge old newsgroups seems not to have been
on anybodies mind. :-(


As for my reason for asking ? I see way to many newsgroup names that are
infantile curses at someone (like over 20 containing "sucks" in their
names). Assuming that once in a while someone over at the newsgroup server
goes over all the "newly-added" newsgroups and deletes those I would like to
get rid of them too (currently I just manually hide them)

Regards,
Rudy Wieser

[DAN]

Post by R.Wieser
Simple question : using the NNTP protocol - How do I check for newsgroups
that have been removed ?

Rudy, your question is incomplete. It should be "How do I check for newsgroups
that have been removed FROM A PARTICULAR NNTP SERVER ?"

And, like others have already described, this isn't part of a general standard.
Including or removing a given newsgroup is a purely local decision.

Therefore, your only hope is either to download the server's new list of groups
and compare it with the old one (winmerge would do the job for you), or to talk
to your local server's admin and ask them whether they maintain a list of
removals and are willing to share it. (I wouldn't hold my breath)

HTH

[Andrzej P. Wozniak]

Post by R.Wieser
Hello all,
Simple question : using the NNTP protocol - How do I check for
newsgroups that have been removed ?

It's very simple. Just subscribe control.rmgroup newsgroup on your
newsserver. Newsgroups control.* are created automatically by every
newsserver, but there may exist some restrictions set by newsadmin:
* control.* newsgroups may be hidden for common users and you can't see
those groups on the list of all newsgroups;
* control messages may expire like other messages and you can't see old
messages.

In such cases you should contact your newsadmin.

[R.Wieser]

Andrzej,

Post by Andrzej P. Wozniak
It's very simple. Just subscribe control.rmgroup newsgroup on your
newsserver.

Thanks, I would never have tought of that.

(doing some googeling in regard to that newsgroup name)

But isn't that group ment for /requests/ for removal (and subject to the
maintainers choices), not the actual deletions (meaning: excluding those
done from the maintainers dashboard) ?

And something else : I just took a peek into that group on ES, and the last
removed entry is of november last year. It also contains names I cannot
remember having ever seen.

IOW, they look to be containing some newsgroup names that ES never offered,
as well as possibly missing locally deleted newsgroups, as they where not
requested by other newsgroup servers.

... Which would mean that its not quite fit for my purpose (cleaning up my
local list of newsgroups).

Post by Andrzej P. Wozniak
* control messages may expire like other messages and you can't see old
messages.

Yep. Though in the case of ES the retaining time seems to be at least a
decade (earliest entry is of march 2015). :-)

Regards,
Rudy Wieser

[Andrzej P. Wozniak]

Post by R.Wieser
Andrzej,

Post by Andrzej P. Wozniak
It's very simple. Just subscribe control.rmgroup newsgroup on your
newsserver.

Thanks, I would never have tought of that.

The control.* newsgroups usually keeps only control messages sent to the
specific newsserver - either by the newsadmin or by approved users of that
server (f.e. cancel messages).

(...)

Post by R.Wieser

Post by Andrzej P. Wozniak
* control messages may expire like other messages and you can't see old
messages.

Yep. Though in the case of ES the retaining time seems to be at least a
decade (earliest entry is of march 2015). :-)

E-S in general has 8 year retention.

If you want to know more details, you should better ask your questions
there:
news://news.eternal-september.org/eternal-september.support
It's E-S local newsgroup which should be available only on E-S newsserver.
Note that for NNTP "local" means group in a local hierarchy (like the
linked e-s one) or local newsgroup distribution (set by "Distribution"
field in a message header).

If you want to talk about newsgroups subscribed in your newsreader, you
should rather use term "offline".

[R.Wieser]

Andrzej,

Post by Andrzej P. Wozniak
The control.* newsgroups usually keeps only control messages sent to the
specific newsserver -

You mean, that subtree is not replicated to all the other newsgroup servers
? Any idea why not ?

Post by Andrzej P. Wozniak
either by the newsadmin or by approved users of that server (f.e. cancel
messages).

Although I took the latter part for granted, I mentioned something in
relation to the former (dashboard). And as that has not been adressed ...

Post by Andrzej P. Wozniak
E-S in general has 8 year retention.
If you want to know more details,

No. If its supposed to have a specific behaviour its likely in an RFC
somewhere. If its not than the info would not be worth anything in regard
to any other newsgroup server.

Ah yes : last mentioning (that I have) in RFC 5537, chapter 5.2 - Group
Control Messages.

And that also answers my above "any idea why not?" question, as it looks
like that they are indeed replicated to other servers too. Which makes
sense.

And as it doesn't seem to mandate that all newsgroup deletions /must/ be
going thru the "rmgroup" newsgroup ... I still cannot use it to (fully)
clean-up my local newsgroup list.

Post by Andrzej P. Wozniak
If you want to talk about newsgroups subscribed in your newsreader, you
should rather use term "offline".

:-) Offline ? Couldn't there be a more meaningless name for it ?

I referred to those lists as "local" and "server" from a users perspective
(which is what I am and the POV I wrote my message from). But if someone
wants to refer to them regardless of perspective than I suggest naming them
"server" and "client" (newsgroup) lists. No ambiguity, no matter which POV
is taken.

Regards,
Rudy Wieser

[sticks]

Post by R.Wieser
Andrzej,

Post by Andrzej P. Wozniak
The control.* newsgroups usually keeps only control messages sent to the
specific newsserver -

You mean, that subtree is not replicated to all the other newsgroup servers
? Any idea why not ?

Post by Andrzej P. Wozniak
either by the newsadmin or by approved users of that server (f.e. cancel
messages).

Although I took the latter part for granted, I mentioned something in
relation to the former (dashboard). And as that has not been adressed ...

Post by Andrzej P. Wozniak
E-S in general has 8 year retention.
If you want to know more details,

No. If its supposed to have a specific behaviour its likely in an RFC
somewhere. If its not than the info would not be worth anything in regard
to any other newsgroup server.
Ah yes : last mentioning (that I have) in RFC 5537, chapter 5.2 - Group
Control Messages.
And that also answers my above "any idea why not?" question, as it looks
like that they are indeed replicated to other servers too. Which makes
sense.
And as it doesn't seem to mandate that all newsgroup deletions /must/ be
going thru the "rmgroup" newsgroup ... I still cannot use it to (fully)
clean-up my local newsgroup list.

Post by Andrzej P. Wozniak
If you want to talk about newsgroups subscribed in your newsreader, you
should rather use term "offline".

:-) Offline ? Couldn't there be a more meaningless name for it ?
I referred to those lists as "local" and "server" from a users perspective
(which is what I am and the POV I wrote my message from). But if someone
wants to refer to them regardless of perspective than I suggest naming them
"server" and "client" (newsgroup) lists. No ambiguity, no matter which POV
is taken.
Regards,
Rudy Wieser

I am wondering why your ? always has that space between the last letter
and it's placement? Even the subject of this thread has it?

[R.Wieser]

Sticks,

Post by sticks
I am wondering why your ? always has that space between the last letter
and it's placement? Even the subject of this thread has it?

Simple answer ? Because I think it aids in easy recognision of the word just
before it.

But a reverse question : why do you guys always seem to preceede the word
"too" with a comma ? Like "lets do that, too". Whats the rub with that ?
I do not see any reason for, or function of it.

Regards,
Rudy Wieser

[sticks]

Post by R.Wieser
Sticks,

Post by sticks
I am wondering why your ? always has that space between the last letter
and it's placement? Even the subject of this thread has it?

Simple answer ? Because I think it aids in easy recognision of the word just
before it.

OK. I was wondering if it was a coding problem of some sort, or an
intentional error on your part. Thanks

Post by R.Wieser
But a reverse question : why do you guys always seem to preceede the word
"too" with a comma ? Like "lets do that, too". Whats the rub with that ?
I do not see any reason for, or function of it.

It is not necessary in your sentence example, but it can be used for
context. There are situations where it is necessary, but I'm sure you
already know that.

sticks

[R.Wieser]

Sticks,

Post by sticks
OK. I was wondering if it was a coding problem of some sort, or
an intentional error on your part. Thanks

Ah, thataway. Nope, I still write posts without the aid of a syntax checker
or similar. But you /could/ regard my using a space infront of a question
(and exclamation) mark as an "intentional error". :-)

Post by sticks
It is not necessary in your sentence example, but it can be used for
context. There are situations where it is necessary, but I'm sure you
already know that.

Thats the thing. Although I can imagine that an extra comma could be
/sometimes/ helpfull (the famous "lets eat grandma" vs "lets eat, grandma"),
I've been reading epub books and I see it /everywhere/ infront of that word.

Regards,
Rudy Wieser

[Ken Blake]

Post by R.Wieser
Sticks,

Post by sticks
I am wondering why your ? always has that space between the last letter
and it's placement? Even the subject of this thread has it?

Simple answer ? Because I think it aids in easy recognision of the word just
before it.
But a reverse question : why do you guys always seem to preceede the word
"too" with a comma ? Like "lets do that, too". Whats the rub with that ?
I do not see any reason for, or function of it.

I would not normally talk about commas, especially in this newsgroup,
but since you did, you should have put a comma after "or function of."

And "lets" should be "let's" and "whats" should be "what's."

[R.Wieser]

Ken,

Post by Ken Blake

Post by R.Wieser
I do not see any reason for, or function of it.

...

Post by Ken Blake
I would not normally talk about commas, especially in this newsgroup,
but since you did, you should have put a comma after "or function of."

The parts are ment to be of equal weight. And in that case, as far as I
know, a comma is not called for.

Post by Ken Blake
And "lets" should be "let's" and "whats" should be "what's."

Well, you got me there. I never can remember when those quotes are supposed
to be used, and even less when they should appear on the end of a word.

Regards,
Rudy Wieser

[Ken Blake]

Post by R.Wieser
Ken,

Post by Ken Blake

Post by R.Wieser
I do not see any reason for, or function of it.

...

Post by Ken Blake
I would not normally talk about commas, especially in this newsgroup,
but since you did, you should have put a comma after "or function of."

The parts are ment to be of equal weight. And in that case, as far as I
know, a comma is not called for.

It is called for. "Or function of" is a parenthetical expression and
parenthetical expressions need to be set off by a *pair* of
parentheses, commas, or dashes.

See, for example,
https://www.yourdictionary.com/articles/parenthetical-expressions-types-usage

or https://grammarist.com/grammar/parenthetical-phrases/

Post by R.Wieser

Post by Ken Blake
And "lets" should be "let's" and "whats" should be "what's."

Well, you got me there. I never can remember when those quotes are supposed
to be used, and even less when they should appear on the end of a word.
Regards,
Rudy Wieser

[Char Jackson]

Post by Ken Blake
I would not normally talk about commas, especially in this newsgroup,
but since you did, you should have put a comma after "or function of."
And "lets" should be "let's" and "whats" should be "what's."

Welcome back, Ken. I was just wondering where you'd gone. :)

[Ken Blake]

Post by Char Jackson

Post by Ken Blake
I would not normally talk about commas, especially in this newsgroup,
but since you did, you should have put a comma after "or function of."
And "lets" should be "let's" and "whats" should be "what's."

Welcome back, Ken. I was just wondering where you'd gone. :)

Thanks. I've had some serious health issues. I'm still not 100%, but
I'm much better.

[Char Jackson]

Post by Ken Blake

Post by Char Jackson

Post by Ken Blake
I would not normally talk about commas, especially in this newsgroup,
but since you did, you should have put a comma after "or function of."
And "lets" should be "let's" and "whats" should be "what's."

Welcome back, Ken. I was just wondering where you'd gone. :)

Thanks. I've had some serious health issues. I'm still not 100%, but
I'm much better.

Good luck with the recovery!

[Ken Blake]

Post by Char Jackson

Post by Ken Blake

Post by Char Jackson

Post by Ken Blake
I would not normally talk about commas, especially in this newsgroup,
but since you did, you should have put a comma after "or function of."
And "lets" should be "let's" and "whats" should be "what's."

Welcome back, Ken. I was just wondering where you'd gone. :)

Thanks. I've had some serious health issues. I'm still not 100%, but
I'm much better.

Good luck with the recovery!

Thanks again.

[Zaidy036]

Post by R.Wieser
Hello all,
Simple question : using the NNTP protocol - How do I check for newsgroups
that have been removed ?
I don't really want to go thru the list of all newsgroups I currently have
stored and check, one by one, if I they are still present on the newsgroup
server.
Remark: I found the command to find all *new* newsgroups (since a certain
date), but have not been able to find its opposite.
Regards,
Rudy Wieser

In a backup just search for news.<newsprovider>.org.rc

[R.Wieser]

Zaidy036,

Post by Zaidy036
In a backup just search for news.<newsprovider>.org.rc

I'm sorry, but I have absolutily no idea what you are talking about there,
what backup (of what) do you mean ?

And to forgo any confusion about it, I'm talking from the perspective of a
newsgroup *client*.

Regards,
Rudy Wieser

[Zaidy036]

Post by R.Wieser
Zaidy036,

Post by Zaidy036
In a backup just search for news.<newsprovider>.org.rc

I'm sorry, but I have absolutily no idea what you are talking about there,
what backup (of what) do you mean ?
And to forgo any confusion about it, I'm talking from the perspective of a
newsgroup *client*.
Regards,
Rudy Wieser

Any image or other backup.
news.<newsprovider>.org.rc contains message status

----Partial sample of mine in a text editor:
comp.mobile.ipad: 1-84069,84070
misc.phone.mobile.iphone: 1-161615,161616
alt.comp.microsoft.office: 1-160
alt.comp.software.financial.quicken: 1-19220,19221
alt.msdos.batch.nt: 1-17038,17039
alt.windows7.general: 1-197490,197491
microsoft.public.excel: 1-27922,27923
microsoft.public.windows.64bit.general: 1-9155
microsoft.public.word.vba.general: 1-12435
alt.comp.os.windows-10: 1-163979,163980
alt.comp.software.firefox: 1-3855,3856
alt.comp.software.thunderbird: 1-4075,4076
alt.comp.os.windows-11: 1-3553,3554

[R.Wieser]

Zaidy036,

Post by Zaidy036
Any image or other backup.

So, just any random backup ? I guess I could grab my backups of my C89
installation disks ...

Again, a backup of *what*. I have zero clue what you are talking about.

Post by Zaidy036
news.<newsprovider>.org.rc contains message status

Is that a newsgroup ? If so, I have not been able to find it in my
newsgroup reader.
[snip]

That looks like the output of the NNTP "groups" command. But if I would
want to go that way I would need a current version of that list, not a stale
one.

Regards,
Rudy Wieser

[Zaidy036]

Post by R.Wieser
Zaidy036,

Post by Zaidy036
Any image or other backup.

So, just any random backup ? I guess I could grab my backups of my C89
installation disks ...
Again, a backup of *what*. I have zero clue what you are talking about.

Post by Zaidy036
news.<newsprovider>.org.rc contains message status

Is that a newsgroup ? If so, I have not been able to find it in my
newsgroup reader.
[snip]
That looks like the output of the NNTP "groups" command. But if I would
want to go that way I would need a current version of that list, not a stale
one.
Regards,
Rudy Wieser

that depends on how many backups you keep for how far back.

[R.Wieser]

Zaidy036,

Post by Zaidy036
that depends on how many backups you keep for how far back.

Ah, you're not here to help, but just to troll about. Alright, than I will
leave you at it.

Goodbye.

Regards,
Rudy Wieser

[Zaidy036]

Post by R.Wieser
Zaidy036,

Post by Zaidy036
that depends on how many backups you keep for how far back.

Ah, you're not here to help, but just to troll about. Alright, than I will
leave you at it.
Goodbye.
Regards,
Rudy Wieser

I am not trolling but just trying to suggest a method to meet your
desire "going forward".

If you do not have the file in an old backup at least add it to your
backup now so in the future you can compare the current accounts list to
what is in the backup.