We regularly encounter scenarios where attackers gain unauthorised access to users’ mailboxes. The aim of the digital forensic examiner is to understand as much as possible about the breach in order to give to recommend the client which steps to take (how access was gained,  how long was the breach undetected, whether emails were sent, deleted or tampered with). Many data sources can be used to put together the different pieces of the puzzle, but one of the first point of call are the email servers logs.

However, what happens when the email servers are not on-premise, but instead provided by a cloud service? How are logs kept (and for how long)? And what is the impact on Vietnamese companies? Do they use cloud email services? Do they keep emails on-premise?

Worldwide, many organisations opt for cloud based email. Running an on-premise setup (Exchange, Postfix+Dovecot…etc) can be tricky and cumbersome. In most cases, once hardware, software licenses and support costs are added up,  such a setup is only cost effective for large organisations (with over 200+ mailboxes). At a “basic” average cost of $5-$10 per mailbox per month, such setup would cost $1000 – $2000 per month total. In Vietnam, that a budget high enough to re-consider.

Some smaller organisations use a local set of email servers due to:

• Confidentiality and/or data protection: if email is used without encryption (S/MIME, PGP…etc) for confidential or commercially sensitive data, keeping files in the cloud may be seen as high risk.
• Bandwidth: some companies transfer a lot of internal email between employees. If a cloud email service is in use, the emails from one member of staff to another will have to leave the sender via the main internet connection and immediately arrive via the same connection to the recipient. This can be mitigated via local document repositories, but it can add complexity.
• Legacy: The company had local email servers years ago and they are still in use. Migration may be planned in the near future.

Vietnam should be no exception to this. However, how could we be certain?

Companies and organisations in Vietnam tend to use the .vn top level domain to signal local presence. This domain is managed by VNNIC. So we could simplify this question by asking “how many of these .vn domains use cloud email providers?”

Now, nor VNNIC, nor most of the other country top level domains, open their record databases (for obvious confidentiality reasons). Some marketing providers sell domain lists for every country on the web, but those lists are normally obtained by scanning servers and other records (like names on SSL certificates). Another option would be to use the project Sonar databases (https://scans.io/) and extract all the .vn domain names. But this is unlikely to give us more than 25% of available domain names. Also, it would not be a fair sample, as organisations using cloud services are on the whole unlikely to run other private services or buy any SSL certificates. They might have chosen to run cloud email to offload as much IT as possible.

But in this particular case, we are in luck. We have a better dataset available at the Hydraski blog (https://hydrasky.com/network-security/over-1-4-million-domain-vn-leaked/). It all started with a brief misconfiguration on one of the VNNIC servers in August last year. From their blog:

“On Aug 31, 2017 at approximately 4:14AM PDT (earliest known detection), one of Vietnam’s top level nameservers was accidentally configured to allow global DNS zone transfers. This allows anyone who performs an AXFR (zone transfer) request to the country’s f.dns-servers.vn nameserver to get a copy of the nation’s top level DNS data. This was detected by the TLDR Project – an effort to attempt zone transfers against all top level domain (TLD) nameservers every three hours and keep a running Github repo with the resulting data.”

So, due to a brief problem on f.dns-servers.vn (one of the two HCM DNS nodes run by VNNIC – https://www.vnnic.vn/en/dns-vnix/map?lang=en), a full set of domain names for all Vietnamese DNS zones was made available. The dataset was archived by one of the TLDR project developers in Github (https://github.com/mandatoryprogrammer/VietnamDNSLeak). We took that as a base for our analysis.

That dataset contains 625892 distinct domain name under the vn top level domain. The subdivision on secondary domains (com.vn, edu.vn…etc) is as follows:

Vietnam second-level domains

The vast majority of domains (like ours at https://www.dfir.vn) are on the vn zone directly. Some companies also register the .com.vn (to protect their brand), schools and universities use .edu.vn and very few use .net.vn and .name.vn. Beyond those zones the number of organisation using other second level domains (like the ones seen per city) are very small (below 1% of the total) and mostly linked to the Vietnamese government institutions.

The next step was to use that dataset to run a small survey to obtain the primary mail server for each domain listed. The list of scripts and raw results used are in one of our Github repositories (https://github.com/richiemann/VietnamDNSLeak). We scanned all the ~600k domains and asked for the highest priority mail server. The highest priority mail server will (in our view) indicate the primary email provider (as that is where most emails will arrive in real-life scenarios).

We got the following raw results:

• Of the 625862 only 171,505 domains (27%) have a working email records. A lot of domains seem to be just defensive registrations to protect brands, or domain squatters (that have no intention of using the domain).
• The top 25 email providers amount for almost one third of domains recorded  (30.91%) in the test. The rest are mostly self-hosted.
• Of the top 10 providers, Google Apps has as much market share as the subsequent 24 providers.

Top 10 Cloud Email services

It is clear that Google Apps is so far a great success in Vietnam, with Zoho a very significant but distant 2nd.

However, how does cloud email usage affect logging for forensic purposes? Do these providers maintain enough information in the event of breach? Can these cloud services be reconfigured or enhanced to store additional information? We will be covering these in further articles.

Part 1 – Google cloud email (G Suite)

Part 2- Zoho email