Ehlo Tech - Focusing on Exchange Server & Technology

Easily Upgrade your BlackBerry 10 Devices to Natively Install and Run Android Apps

noreply@blogger.com (Ben Serebin) — Fri, 03 Jan 2014 15:21:00 +0000

For you BlackBerry power users (I'm a Q10 user) looking to run Android apps, this is how to do it (there was a single app I wanted which is a train commuter scheduler [RailBandit]). The newest leaked Blackberry OS includes Android runtime 4.2.1 and allows ability to natively load Android apps (yes - you load apps right from your BB - AWESOME!). Sadly, Verizon hasn't launched the next OS (10.2 even though it's been out for MONTHS), so you can do it yourself. I've been running this for 12 hrs, and so far I like it. It's super easy as well. This is to run the brand new leaked 10.2.1.1925 (radio 1899) less than a week old. What's cool about BB10 is you can upgrade the radio code separately from the general OS. Makes it easier for the carrier to test and release updates (or in theory should). The juicy details...

Use This to Download Your OS (Q10, Z10, Z30, etc)
http://forums.crackberry.com/bb10-leaked-beta-os-f395/lengend-presents-10-2-1-1925-all-q-z-887047/

Backup Source
http://www.bbin.in/en/2013/12/24/leaked-os-10-2-1-1925-for-all-blackberry-10-smartphones/

Here are the steps to install a leaked OS on your BlackBerry 10 device (most popular desktop OS required):

Download and install BlackBerry Link (latest is 1.2.2.13 b122) direct from BB.
Backup your BB, just in case and to downgrade if you need to.
Download correct OS from CB URL link above (don't worry about the term "radio" - from the one that says "10.2.1.1925").
Switch off your BlackBerry 10 device
Run the AutoLoader File
Connect your BlackBerry 10 Device to your PC via USB
Quickly Turn on your BlackBerry 10 device
The AutoLoader will detect your device and starts the installation
Please hold till the AutoLoader reaches to 100% and CLOSES automatically
Disconnect your BlackBerry 10 device and wait till it starts again
Security wipe before start using/restoring your data (I didn't do this, but the instructions suggest it)
Manual Setup your config, apps, etc
Enable the install apps functionality from 3rd parties. The first time you download and run an app, the OS will prompt you to enable this feature. Thanks BlackBerry!

Bonus: 2 good sources for loading Android apps is Amazon Apps and 1mobile.com. Load both App Store Apps and you can load your new Android apps easily.

Screenshots of the goodness (note: no BES required, but I run one since I roll like that)....

Before

After - 10.2.1.1925 (yeah baby)

To a Happy, Healthy, Productive, and Efficient 2014!
-Ben

Network Solutions = AOL Email. If you want to be respected in the 21st century switch to another provider.

noreply@blogger.com (Ben Serebin) — Mon, 18 Nov 2013 15:27:00 +0000

Worked on a server environment migration to a datacenter yesterday. Everything went smoothly except for DNS was not working for the client. I spent an extra 2 hours troubleshooting DNS. I repeatedly attempted to update DNS in the Network Solutions webpage, and it would not show the updated entries. Tried with IE, Chrome, and multiple OSes. Nada. Turns out it was a bug in the NetSol system. A call to support indicated that. So, I was assured it would be resolved on Sunday, almost 24 hrs later on Monday, still not working. DNS A records are showing up web page with one entry, and the result is not showing up on the NetSol NS query responses. So, if you updated with 1.1.1.1 and do a DNS query the original IP pre-Sunday.

Just to be CLEAR.... if I query ns57.worldnic.com & ns58.worldnic.com which are the client's NS entries PER Network Solutions, they respond with the wrong entries. How nice. Call to support, and the script reading call rep, didn't understand the issue and kept repeating it takes 24 hours to propagate. Another favorite is you delete or change an entry on the NetSol "advanced DNS" page and the changes aren't displayed. Logout and log back in, same info. This happens on the main webpage and DNS as well. What a joke of a system. I tried to switch to a 3rd party DNS system, and the NetSol NS are not following the DNS spec for NS lookups at point of entry (I added my NS entries ns10.operationdns.com, etc). NetSol claimed they cant' be found, but even the native Windows DNS finds them. Sigh.... So sad the founding internet's domain name provider has failed so badly similar to AOL.

DNS Tool I like since it allows me to query against any NS and for any record
http://centralops.net/co/

-Ben

BES 5 & BES 10 Server Consolidation Coming...

noreply@blogger.com (Ben Serebin) — Wed, 27 Feb 2013 17:17:00 +0000

Currently, users running BB 4-7 need BES 5 and BB 10 requires BES 10. So, you need to run 2 separate servers for BES 5 & BES 10. This will be changing once an update is released to BES 10 (ETA is May 2013). This update will be "managed using a single fully featured management console". Then BES 10 & BES 5.0.3+ will be able to run on the same server. Very helpful.

-Ben

Exchange 2010 & 2013 Compatible Not There Yet...

noreply@blogger.com (Ben Serebin) — Wed, 27 Feb 2013 17:16:00 +0000

With the recent release of Service Pack 3 for Exchange Server 2010 compatibility with Exchange Server 2013, we are getting closer, but NOT yet. Now Exchange Server 2013 needs an update. The update is tentatively called Exchange Server 2013 CU1. That will then allow integration between 2010 & 2013. ETA should be available "shortly". I would guess March or April of 2013.

-Ben

Apple Screws Up iOS EAS Implementation AGAIN! - Exchange Suffers

noreply@blogger.com (Ben Serebin) — Wed, 13 Feb 2013 15:59:00 +0000

Has your Exchange 2010 CAS recently started experiencing heavy CPU, insane transaction logs growth (aka DoS), Apple might be the cause. Apple released iOS 6.1 last week (beginning of Feb) and it has a "slight" EAS bug in it. ;-) Another bonus... it drained iOS batteries and wasted data bandwidth. So bad... Vodafone txted all iPhone users to NOT upgrade to 6.1. Oops.... and 6.1.1 doesn't fix it...

You would think a large company (aka Apple) that prides itself on releasing "amazing products" would not repeat their mistakes in the past (iOS 4), but sadly history is repeating itself. iOS 6.1.1 fixed another problem. This is why I am not a fan of EAS. It's buggy since vendors don't have a real incentive or pressure to make sure EAS works well. Sure, Microsoft releases "guidelines" and crosses it's fingers vendors follow, but it's a frequent problem. I know, since I see the EAS connections logs and the Exchange problems they cause. I'll post in the future, how to look this up in your Exchange logs. I'm leaving EAS soon enough... I'll be happy to return to the BlackBerry ecosystem when the Q10 is released. That's another post...

See the Microsoft KB article, but to summarize, there are is a quick fix:

- delete the Exchange account from the iOS device and re-add

iOS 6.1 EAS Bug - Workarounds

http://support.microsoft.com/kb/2814847/en-us?sd=rss&spid=13965

iOS 4 EAS Bug (walk down memory lane)

http://blogs.technet.com/b/exchange/archive/2010/07/01/3410271.aspx

-Ben

Exchange Book Review - iPhone and Exchange Server 2010

noreply@blogger.com (Ben Serebin) — Mon, 25 Jun 2012 14:13:00 +0000

Hello All,

I was asked to review an Exchange book with a focus on iOS by Steven Goodman. Overall good Exchange Server book with decent amount of iPhone documentation. But there is a lot of non-relevant (non-iOS) content to call it an iPhone & Exchange book. I would refer to it as an Exchange Server Setup with a focus on iOS functionality.

There's a lot of content, which isn't relevant to experienced admins looking to brush up on iOS functionality/capabilities. For example, who needs to know how to run the "setup.exe" on Exchange (w/screen shots), design planning, HA, etc. Come on! Focus on iPhone integration. So, skip to the chapters you want to read, and don't attempt to read this cover to cover. The book lacks some important information about iOS devices such as throttling policies for iOS/ActiveSync devices, reviewing logs to determine performance usage (1/2 page on analyzing reports isn't adequate), etc. Don't get me wrong, the book is a good Exchange 2010 book with touches of iOS knowledge. But, if you run into Exchange performance issues with iOS/ActiveSync devices, you'll need more technical knowledge than this book offers.

-Ben

P.S. I agreed with the other review that the author is jumping on the "Apple bandwagon" by titling it as an iPhone Exchange Server book.

Our missing email admin friend called telnet!

noreply@blogger.com (Ben Serebin) — Fri, 16 Mar 2012 14:36:00 +0000

Want a free tool to quickly test your email server is responding to port 25 and is acceptable mail. Use the native tool available in Windows. Sadly, it's not installed by default, so you'll need to add it as a "Feature". And then you can do things like telneting to port 25 and see your email server respond.

See this easy to follow Microsoft article to walk you through the telnet process to send email. I use this often. You can easily find it by googling "telnet port 25" and then bang, you have access to it wherever you are. No need to memorize the steps. http://support.microsoft.com/kb/153119

-Ben

NYExUG 3/12 Meeting Follow-up - Troubleshooting Tips

noreply@blogger.com (Ben Serebin) — Fri, 16 Mar 2012 14:23:00 +0000

We had an excellent Exchange User Group meeting this past Tuesday about Troubleshooting Tips. Even though we ran later than we had in a while, we could not cover everything. I've highlighted comments and feedback received during the meeting and after. Thank you to everyone for your feedback. This is what makes us a community.

NYExUG Exchange Troubleshooting and Tips Presentation

Correction: the Exchange Server User Monitor (ExMon) tool does not list ActiveSync versions, but other AS performance stats.

Dirt cheap $60 UCC/SAN (5 names) certificate I recommend is https://certificatesforexchange.com/ which is backended via GoDaddy.

RDP Manager I use is called RoyalTS ($35) which has a lot of flexibility, functionality, stability, and works on XP and above. http://www.code4ward.net/main/

Website tool highlights from my presentation

Microsoft owned Remotely Testing Outlook (RPC), ActiveSync, Web Services, Email (smtp) and even Office 365 https://www.testexchangeconnectivity.com
Review DNS Records for Auto Discovery - http://centralops.net
Review DNS Records for basic MX/A Records – http://tools.appriver.com
Checking for RBLs – http://www.mxtoolbox.com
[application] Stress testing application for generating email (Server Traffic Test) - http://www.icewarp.com/downloads/tools/ No reboot and stable application.

Attendees Feedback (thank you)

The website designed to check your TLS configuration. http://www.checktls.com
"Out of control transaction logs. On a number of occasions we have had the transaction logs grow significantly (one every second or so). This can be caused by a rogue application sending emails via your HT or a bad out of office configuration. We have experienced both. The last one was a user's out of office settings. They had used the rules section in the out of office. Viewing the transaction log showed the user forwarding the same email every second or so. Turning of the out of office resolved the issue."
Exchange environment summary report based (# of Exchange Servers & mailboxes, DB sizes, DAG status, etc. http://www.stevieg.org/2011/06/exchange-environment-report
Post on troubleshooting ActiveSync issues from the Exchange Team Blog. http://blogs.technet.com/b/exchange/archive/2012/01/31/a-script-to-troubleshoot-issues-with-exchange-activesync.aspx
Tony Redmond wrote an excellent article about ActiveSync not working as a result of 2010 user being a member of the priv’ group on his blog site. If he’s truly 2003 user, then the only thing I can think of is setting up similar profile on a different iphone. If that stil doesn’t work then it’s the account & he may need to look into deleting the EAS association via adsiedit & redo the EAS profile on the device. An Exchange MVP (Michael B Smith) has commented several times in the past on the MSExchange forum re: the ills people have been experiencing with iphones – in our own environment we’ve seen disappearing emails/corrupted calendars/and all sorts of wackiness. I can forward forum posts if people are interested but my desktop team has been beaten into submission about what to do/not to do when it comes to syncing EAS devices with Exchange. http://thoughtsofanidlemind.wordpress.com/2010/10/08/ex2010-insufficient-access/
Free Microsoft RDP manager mentioned was Remote Desktop Connection Manager (I didn't like the last version, so I know nothing about this one -Ben) http://www.microsoft.com/download/en/details.aspx?id=21101

Any comments, post them or email me. Thanks.
-Ben

HP SAN vs "Dividing by Zero". 0-1. SAN hardware crashes.

noreply@blogger.com (Ben Serebin) — Mon, 13 Feb 2012 04:36:00 +0000

Hello All,

Never, ever, ever, ever divide by zero otherwise very bad things can happen. A client's less than 1 year old HP SAN environment crashed last week thanks to a SAN firmware bug (dividing by zero) which caused a kernel panic. And of course this only happened when uptime hit 208.5 days. Hence, HP calls it the "208 Day" bug. I call it poor software development.

Absolutely ridicuous. I have never really liked the HP SAN hardware which was brought over from Lefthand Networks. For mission critical environments, I'm a believer in proper SAN hardware such as the Dell EqualLogic line. REEF has deployed HP and Dell SAN hardware, and without a doubt, the Dell SAN hardware is better. Even the HP software has problems with Hyper-V and running under Windows Core. Disappointing. The Dell SAN hardware is better built and cheaper, and this is why REEF Solutions' is a Dell Premier Partner.

HP bug which causes reboots after 208.5 days

http://h20000.www2.hp.com/bizsupport/TechSupport/Document.jsp?lang=en&cc=us&taskId=110&prodSeriesId=4118659&prodTypeId=12169&objectID=c03090774

-Ben

P.S. The IT Director of the client who experienced the problem at least has a good sense of humor. This "dividing by zero" programming mistake is clearly a common issue. Enjoy the image below.

Preparing for D Day for Me...

noreply@blogger.com (Ben Serebin) — Sun, 12 Feb 2012 16:22:00 +0000

Hello All,

I've been doing a lot of "house keeping" lately before "D" Day. "D" day being delivery day. My wife is due with our 3rd child. While my wife is nesting, I'm doing the equivalent for an IT person. We had a false alarm when we thought it was happening, so now I feel like I'm living on borrowed time and have all this "extra" time. In the last week I've done the following:

getting our REEF NY & TX SonicWall firewalls updated to the latest code (VPN tunnel speed to my TX off-site environment doubled in speed)
rolling out a SonicWall based network bandwidth and auditing solution (we currently monitor it using another solution) for REEF's networks.
NY based on-site servers replication operating system re-installed (for REEF environment, the on-site server is 2008 R2 based. The replication data was not touched, since it is iSCSI based.)
NY based on-site servers replication software upgraded (to improve performance, noticeable positive difference between AppAssure Replay 4.6.1,31257 and 4.7.2.40512 [found a bug in the replication UI and alerted AppAssure about it and received a support response in 5 minutes. Impressive. I wish all AppAssure support techs responded so quickly]). For REEF environment. Enjoy the image below.
TX based off-site servers replication upgraded (same AppAssure Replay versions upgraded)
rolled out my digital photo album solution based on a BlackBerry PlayBook. Considered an iPad, but security, performance, and low cost of the 64GB PlayBook ($300) made it the better solution.
NY on-site server operating system re-installed (for clients environment, the server environment is Windows 2003 x86 based. Currently using a stable release of Ahsay. Planning to upgrade to latest stable version shortly.
working on deploying a new wireless SonicWall based solution so guests at home will be on a separate VLAN based network. In preparation for all those home visitors.

Notice the replication speed showing “10.22MBit/sec”. It should be “Mb”, not MB. A capital “B” is BYTES, while a lower case “b” is bits. This is on the latest version 4.7.2.40512. Dev has been alerted per support's response.

Back to spending time with the existing kids and wife,
-Ben

Recommended Exchange Deployments are Multi-role Now

noreply@blogger.com (Ben Serebin) — Sat, 24 Dec 2011 06:49:00 +0000

At the last free "Tech Ed style" event in NYC held at the Microsoft Offices, we had Ross Smith IV present on

Exchange 2010 High Availability/Database Availability Groups. If you don't know Ross, he's a VERY senior Microsoft employee who wrote the Exchange Storage and Server Role Calculator. He knows Exchange, period. End of story. So, when he said that everyone should deploy Exchange 2010 in a multi-role configuration to improve performance and not break apart the roles, you need to take his recommendation seriously. This was the 1st time I had heard this. I had a long conversation with him about this in NYC, and he explained that for performance and the ability for leveraging failover capacity it is better to keep all the roles together. In theory, you could deploy less. Since if you were going to deploy 2 CAS and 2 Mailbox, you could in theory just deploy 3 consolidated roles. Well, Microsoft TechNet finally released some guidance on this. That only took 6 months. Don't forget to use a hardware/VM load balancer when deploy your multi-role Exchange Servers.

TechNet article title: Understanding Multiple Server Role Configurations in Capacity Planning
http://technet.microsoft.com/en-us/library/dd298121.aspx

-Ben

Hackers & Malware - Dangers Everywhere - Not Just Scare Tactics

noreply@blogger.com (Ben Serebin) — Sat, 24 Dec 2011 05:10:00 +0000

Hackers and malware were busy this week at clients of REEF Solutions.

Good news first, we identified a serious denial of service vulnerability during a network infrastructure review for a financial firm. So bad, a simple command would reboot a core network device. That was a highlight of the review. And this was not even a security audit, I am sure it'll be only worse.

Bad news now

A client's system was infected with TDSS, one of the nasty [known] malware products (think encryption, p2p command and control, http/https tunneling, malware competition removal, and MBR infection). Malware vendors even offer a Firefox plug-in to allow paying customers to surf via infected machines to provide anonymous cover. To summarize, TDSS is extremely dangerous. More technical details here. As of now, the only tool that can remove it or most of it is Kaspersky. Ideally, we should have wiped the system, but the client would not permit this.
A hacker attacked via RDP and compromised a system. We detected the compromise and took immediate action to isolate and remediate the attack. If we had not caught it faster, this could have been a serious issue. The key is to have an Intrusion Detection System in place, even if it's just a firewall based solution. You need to be aware of what is happening on your network. I recommend additional policies such as resetting all administrator passwords, not permitting "administrator" usernames, requiring 15+ characters passwords, email alerting w/3rd party logging tool on administrator level logins, and layered security products (firewall based scanning, servers based, proxy based, DNS scanning, etc).

Sadly, malware and attackers are not sitting idly by. There are some real threats out there. Stay safe...

-Ben

Sprint 4G appears to be hacked at DEFCON

noreply@blogger.com (Ben Serebin) — Fri, 12 Aug 2011 17:17:00 +0000

Hello All,

It appears there could have been a successful man in the middle attack (MiTM) on Sprint 4G at DEFCON. Numerous Android devices were attacked during this period. I hope Android users didn’t "upgrade" or re-enter "their passwords" during the multiple day event. Dangerous, but there is a solution that the carriers and handheld manufacturers could implement to protect against this (see solution below).

News from:
http://www.extremetech.com/computing/92370-4g-and-cdma-reportedly-hacked-at-def-con
http://seclists.org/fulldisclosure/2011/Aug/76

A friend and I were discussing this and this what his response was:

I'm betting that they did it one of two ways on 802.16/ClearWire/Sprint4G.

They gained physical access to the local tower, and did MiTM from the tower. WiMax is Mobile IP from the tower to the provider edge. It would be a lot easier to do MiTM on at a Mobile IP tower, rather than a LTE network.

The other way is, someone in the group worked for / had access to enough of the parts to make a fake WiMax base station. Based on the signal strength reports, and slow speeds, this is what I bet they did. WiMax uses either EAP-TLS or EAP-TTLS. I'm guessing either they had access to the certs to appear valid, or the end devices did not properly implement EAP-TLS and EAP-TTLS, and just accepted any certificate

A pretty cool hack if they did. Hopefully it can be shown, and the 4G devices can implement proper security.

Update to above.....

After further research, it definitely looks like the latter method (fake WiMax Base station). They talk about signal strength and upload speeds. Those wouldn't be affected by the first method (getting into a valid tower).

Unfortunately the supplicate (client) is probably just configured to accept any client.

For example in the Cradlepoint, you just specify the carrier / realm, but that's it. No username, etc. No certs. No other options.

Another example, the Sprint SmartView client, it doesn't have the ability to specify anywhere anything related to authentication and certificates.

One would need a fake WiMax base station (that can do 2.6ghz) in order to test to see if the supplicate takes any certificate.

Side note: certain applications offer the ability to register against a specific TLS certificate serial number such as Apple Mail. I hope other devices/applications allow this in the future.

The MiTM Attack Solution

If phones only accepted carrier based certs and had a proper implementation of EAP-TLS or EAP-TTLS this would protect against this sort of attack.

Sadly, the solution is going to take a bit of work and time. So, don't automatically "trust" voice over data. Protect your data and it can be more secure than your data.

-Ben

New Microsoft ActiveSync Compatibilty Program fails on helpfulness

noreply@blogger.com (Ben Serebin) — Thu, 28 Apr 2011 21:07:00 +0000

Ehlo All,

Curious about knowing...

what ActiveSync functionality is available with which version of Exchange?
which mobile devices have higher Active Functionality?

Well, this new Microsoft ActiveSync compatibility program for OEMs won't help, but read on about it.
Microsoft recently announced the Exchange ActiveSync (EAS) Logo Program for OEMs (think HTC, Google, Apple, Motorola, Microsoft, etc) which should have been used to identify and bring clarify to the level of EAS support a mobile device included. Sadly, it does not do this since there is 1 level for EAS Logo Program and it includes very basic functionality. So, if the device says "ActiveSync", this is pretty much equal to the EAS Logo Program. BK (author of post below) had it right that there should be multiple levels. For example, "basic", "enhanced", "ultimate". So, if an ActiveSync device said "Ultimate", you would know it supports every feature under the sun for EAS against Exchange 2010. Oh well, maybe version 2 of the program will get this improvement.

Windows IT Pro Post about new EAS Logo Program by BK Winstead

-Ben

Message Dehydration isn't a good thing for Exchange!

noreply@blogger.com (Ben Serebin) — Mon, 25 Apr 2011 22:05:00 +0000

Ehlo All,

Quote of the week: "limit is 94% before message dehydration occurs."

A normally very stable client's Exchange Server 2007 stopped processing inbound emails and this was the issue above. Client reported all internal email was working though. I logged into their Exchange Server and reviewed the normal issues inside the Exchange Management Console and nothing jumped out (e.g. databases mounted, 3GB free space on C, 700GB+ free space on database partition (D), no quota limits, receive connectors present/enabled, etc). Strange. I decided to check the email filtering solution in front of the Exchange Server. Since I always recommend clients use my company's email filtering service (SpamCop - I'll discuss this is another post) since it allows me to quickly troubleshoot issues and provide very secure email service. Reviewing SpamCop outbound queue to the client's Exchange Server illuminated the error:

Deferred: : host 55.55.55.55 said: 452 4.3.1 Insufficient system resources (in reply to...

More details about this error can be found on this posting. Checking the client's Application Event Log for this error, I found it "The Microsoft Exchange Transport service is rejecting message submissions because the available disk space has dropped below the configured threshold.". Which was surprising since the C drive had a decent amount of space available (3GB+). What changed? Good old SBS's Windows Server Update Services had downloaded every update under the sun and the storage space threshold was passed and triggered back pressure. I uninstalled WSUS and that eliminated 10GB and back pressure was eased and email flow started. I plan to remove some other functionality as well and plan to do a scheduled reboot as well to free up more space. To sum up, SBS causes more problems than it's worth for Exchange deployments. I prefer clean Exchange installs over Exchange SBS installs.

-Ben

Fixing ActiveSync on an Exchange 2007 Server

noreply@blogger.com (Ben Serebin) — Sun, 24 Apr 2011 05:04:00 +0000

Ehlo All,

So, a client's iPhone with ActiveSync stopped working with their Exchange Server 2007. If rebooting and deleting and setting it back up doesn't fix it, confirm your server's AS is working. An good way to do this is to test with Microsoft Exchange test website found here. So after some investigating, it turned up that the IIS Virtual Directory for "ActiveSync" wasn't responding correctly. One can test this by "https://myservername/Microsoft-Server-ActiveSync" and make sure it prompts for a username and pwd. My client's server didn't do that. It reported 501 service unavailable error. I therefore deleted the ActiveSync virtual directory and re-created it and it fixed the issue. A great blog posting to explain this can be found here. To clarify, the "XXXXX" in his example for normal Exchange installs is "Default Web Site".

Handy URLs:
Detailed instructions on deleting and recreating virtual directories for Exchange 2007
http://my.opera.com/RavenOverride/blog/2009/06/17/how-to-recreate-all-virtual-directories-for-exchange-2007

Microsoft's Test Website for Exchange/ActiveSync/Outlook/SMTP
https://www.testexchangeconnectivity.com

-Ben

Techstravaganza Event with amazing Exchange Speakers (free)

noreply@blogger.com (Ben Serebin) — Fri, 11 Mar 2011 02:46:00 +0000

Ehlo All,

My user group (NYExUG) and 4 other UGs are organizing an amazing event that is a free Tech-Ed type event which has 5 tracks (Exchange, SharePoint, PowerShell, Server/Office, Ask the Experts) and 5 sessions in each.

For those who attended this past week’s NYExUG meeting, I mentioned that at the upcoming Techstravaganza event we might have an Exchange Superstar presenting. Well, that has happened! Ross Smith (Microsoft) will be presenting (thanks to Bob Hunt). You do NOT want to miss this event! Just to give you an idea, this guy frequently presents at major conferences and even runs a session called “Stump the Experts”. With “Stump the Experts”, if you’re able to ask an Exchange question that Ross can’t answer, you win an Xbox. He’s never given away an Xbox. This guy knows Exchange. Period. Learn from a superstar. He’ll be presenting on Exchange 2010 High Availability / DAGs.

Techstravaganza

Held at Microsoft’s NY office

Friday, March 18 (8am-5pm).

This event will have 5 tracks of Exchange, PowerShell, SharePoint, Windows/Client Server, and Ask The Experts.

Register asap since spots are limited

http://techstravanganzanyc.eventbrite.com/

Main Website

http://www.techstravaganza.com

Hope to see you at the event...

-Ben

1st Known Spamming from the "Cloud"

noreply@blogger.com (Ben Serebin) — Tue, 18 Jan 2011 06:59:00 +0000

Ehlo All,

This is my 1st confirmed spam (mid 2010) I have ever seen come from the "Cloud". The "winner" of this honor goes to Amazon. Congratulations (sarcasm). According to my latest research Amazon still does not allow PTR (rDNS) records which is typically required/strongly recommend to avoid outbound email being labeled as spam. How do the many mail servers running on Amazon's Cloud handle it? They relay their email from Amazon's environment onto another host (e.g. authsmtp, google "smtp relay service", etc) and then the other service forwards.

Background on me
I see a LOT of spam since my firm handles filtering for most of our clients via our geographically diverse clustered anti-spam/virus/DoS solution. Our clients on an average day get a total of about 300-400k connections a day (spam/real). This provides me a lot of experience/exposure with spam filtering. If you are wondering why we run our own systems it is because it offers more flexibility, significantly lower latency for email messages (aka delay), and faster response than the big guys.

View from my Spam Filtering Solution which Quarantines Suspect Email Like This.

Spam Header Details
Received: from mm-notify-out-209-61.amazon.com (mm-notify-out-209-61.amazon.com [72.21.209.61])

by mail.rbkgroup.com with ESMTP id 67cz6639988tcu.19.20100625083501;

Fri, 25 Jun 2010 11:35:01 +0200

Date: Fri, 25 Jun 2010 11:35:01 +0200

X-Barracuda-BBL-IP: 72.21.209.61

X-Barracuda-RBL-IP: 72.21.209.61

From: "Buy.com"

Reply-To: Nobody

To:

Message-ID: <02630844.67618272250016768122.JavaMail.em-build@na-mm-relay.amazon.com>

Subject: Thanks for your order!

X-AMAZON-CLIENT-HOST: digital-docs-dope-5002.iad5.amazon.com

X-ASG-Orig-Subj: Thanks for your order!

Bounces-to: 20100625083501q4b3332ggg949lm9p0629fm7g208en6r@bounces.amazon.com

X-AMAZON-CLIENT-SENDTIME: Fri, 25 Jun 2010 11:35:01 +0200

X-AMAZON-MAIL-RELAY-TYPE: notification

X-AMAZON-RTE-VERSION: 2.0

MIME-Version: 1.0

Content-Type: text/html; charset=UTF-8

Content-Transfer-Encoding: 7bit

--------------------------------------------

Any questions, let me know.
-Ben

DoS of DNS by an Exchange Focused Backup Software (AppAssure Replay)

noreply@blogger.com (Ben Serebin) — Sat, 15 Jan 2011 22:29:00 +0000

Ehlo All,

Imagine to my surprise that my favorite Exchange & Windows backup solution (AppAssure Replay 4.5.1.27532) was attempting to cause a denial of service (DoS). This version has a major problem with it's use of DNS lookups within the problem. Within 3 days, one Replay Server had performed over 450,000 queries of the hostname I used for Replay replication. This is almost 100 queries every minute 24 hours a day. That's what the product is doing. This is a serious issue. I've alerted the vendor, so I'm sure a fix will be included in a future release. In the mean-time, see below for the work-around until that happens.

The Issue
Inside AppAssure's Replay for replication, you specify a "Replication Target Host Name". This can be a hostname or IP address. See below for setting within Replay.

This "Select Replication Target" configuration is per protected server (e.g. your Exchange Server, etc). I normally use a hostname for these types of settings since I'm a big fan of using DNS instead of IPs when possible (saves time when changing IPs & saves brain memory space for Exchange Server things). So, when you add your Replication Target hostname, the Replication target and source perform lookups more often than the snap-shot period (x min/hrs). In reality, Replay should only perform a DNS lookup when a replication needs to occur and NOT almost a 100 per minute.

AppAssure's Replay abusing DNS lookups. View from my Firewall hostname query logs.

The Workaround Until a Permanent Fix is Released by AppAssure
If you use a hostname within the Replication option, make sure you add the corresponding information inside the hosts file (c:\windows\System32\drivers\etc\hosts - format is IP address space and hostname - use notepad to open the "hosts" file) on the source AND target Replay Replication Server. This avoids the use of an external DNS query and the query is handled by the operating system. So, this speeds up the process of performing a lookup and reduces your hostname's name server load. Otherwise prepare for your DNS to be attacked by your Replay environment.

Sadly, this isn't the first time I have seen a product mis-use DNS, but it's one of the worst in recent memory.

-Ben

Article - Constitution Finally is Defended over illegal email searches by US Government

noreply@blogger.com (Ben Serebin) — Mon, 27 Dec 2010 05:04:00 +0000

Hello All,

Glad to see someone cover this topic and that this "piece of paper" above means something. "This topic" is about defending our rights (4th Amendment which is unreasonable searches) from the US government illegal intrusions into email. US Dept of Justice illegally had an ISP copy someone's email and was slapped on the wrist, thank goodness for a Court of Appeals. See the Windows IT Pro article from Paul Robichaux discuss this. I hope this ISP get's sued as well now.

-Ben

Article - HA for Exchange 2010 is more costly than Exchange 2007

noreply@blogger.com (Ben Serebin) — Sun, 26 Dec 2010 06:10:00 +0000

Ehlo All,

This is an excellent article especially if you're using or considering deploying Exchange 2007 in an HA configuration with SCR,. Also, the new HA mailbox solution in Exchange 2010 is called Database Availability Groups (aka DAGs). This requires the Enterprise Edition for Windows Server 2008, and we do not know if it requires Exchange 2010 Standard or Enterprise. Either way, the requirement of Server Enterprise Edition makes it costly.

Excellent website (Exchange-Genie.com) that explains with pictures the following HA solutions for Exchange 2007 & 2010, CCR, LCR, SCC, SCR, and DAG. In this case, a picture is worth a thousand words.

Information is power,
-Ben

Intro to Exchange 2007 & SSL Certificates

noreply@blogger.com (Ben Serebin) — Sun, 26 Dec 2010 06:09:00 +0000

[Written in December 2008. Not sure why I didn't post it back then. Found in my drafts.]

Ehlo All,

SSL Certificates (aka SSL certs, certs) are complicated. Exchange is complicated. Quick story... during my research, I read about a Microsoft employee (Exchange admin) who thought a bug existed with a SSL cert vendor's special "Exchange 2007" cert. So he contacted an Exchange Team PM about the issue and it turned out he didn't properly configure the Exchange SSL cert generation. Moral of the story, add SSL certs & Exchange 2007 together, and it can be a real challenge. So, I recommend you read this intro if you're considering or even using Exchange 2007 & real SSL certs especially since many things have changed.

First, the good news with Exchange 2007. Microsoft now includes self-signed certs on install. So, you're secure out of the box. Meaning, everything is encrypted, but you'll get errors each time you access a SSL protected site with it. So, you'll want to fix this.

Well, everything you know about Exchange 2003 SSL certificates can be thrown out, or that's what I've learned so far. You can use wildcard SSL certs, but you'll run into issues in the future (Outlook auto-discovery has issues, Windows Mobile 5 has issues, etc), so I would recommend you embrace the new SSL cert that Microsoft wants you to use OR learn about a special DNS configuration that lets you avoid this (thanks to a MS June 07 update for Outlook 2007).

As per the new certs, they are best described by DigiCert (I liked their explanation & enhanced it some) is:

Exchange 2007 Certificates Definition
The new SSL certificates are called Unified Communications Certificates (aka UC Certificates, UCC Certificates, or SAN certificates) which give you full control of the Subject Alternative Name field so you can secure as few or as many host names as you like with just one SSL certificate. These are NOT wildcard certs since they secure specific hostnames you define within this one cert. Wildcard certs secure any subdomain (e.g. *.mydomain.com), while UCC certs secure (autodiscover.mydomain.com, webmail.mydomain.com, mail.mydomain.com, exchsrv.mydomain.local, etc).

Microsoft's Recommended List of UCC Cert Providers as of 12/08
Comodo - http://www.comodo.com/msexchange/index.html
DigiCert - http://www.digicert.com/unified-communications-ssl-tls.htm
Entrust - http://www.entrust.net/ssl-certificates/unified-communications.htm
Microsoft's Recommended List of UCC Cert Providers

Special DNS Configuration to Avoid UCC Certs
So, you decided you didn't want to spend about $250-300 on a UCC cert for one year. I can understand. There is another option I hinted to above that relates to a June 2007 Outlook 2007 update that added a special feature to avoid the need for a UCC cert for autodiscovery or complex admin configuration. It involves setting a special DNS record to get around this. The DNS record is a SRV record. Once you have this SRV record set, the Exchange 2007 server's externl adn internal URLs need to be this one server as identified in the SRV DNS record.

Learn more about this in Microsoft White Paper on Auto-Discovery in Exchange & Outlook 2007. http://technet.microsoft.com/en-us/library/bb332063.aspx

MS Article on how to setup DNS SRV record for auto-discovery functionality
MS KB article about auto-discovery issues and fixes

-Ben

"Cheating" on an Exchange 2003 Hardware Upgrade

noreply@blogger.com (Ben Serebin) — Sun, 26 Dec 2010 06:08:00 +0000

[Project completed in June 2008. Figured this was in my drafts, I should post it.]

Hello Everyone,

I "cheated" on an Exchange 2003 hardware upgrade I did last weekend (Fri-Sat), or that's at least how I feel since this was hands down the fastest and easiest upgrade I've ever done (and it was about 80GB of db's on an older server with direct attached storage). At the end of the weekend, I started to think maybe I should carry around one of these "things" for my clients for upgrades. I'll share what this "thing" was later in the posting. I don't want folks to think I'm pushing products. My role in the project was to insure the replacement of the Exchange Server hardware went smoothly. The client was in production 24/7 and literally the office was staffed 6 days a week. So, I was concerned originally how to insure minimal downtime.

Background on existing hardware & performance
We were upgrading from an Exchange 2003 Server that was installed with 3 hard drives in a RAID 5 hard drive configuration (direct attached storage) for the OS, transaction logs, and Exchange databases. Company had about 60 users and 30 BlackBerrys or so. 1 BES user adds a load similar to 2 Outlook users. So, total company usage was about 120 users. Performance was an issue, so some users were configured for cached mode to "improve" performance. Cached mode should not be required for LANs, unless Outlook end users are receiving "retrieving data from network". Recommended another DAS server that used RAID 1 for the OS, RAID 1 for transaction logs, and RAID 10 for the Exchange databases.

The Migration
So, I checked the OS install (another admin handled that), Exchange install (using the /disasterrecovery switch for setup and service pack 2), Exchange config, and insuring the email & public folder migration completed successful. Only catch was during this server replacement, there was to be no downtime and no use of Exchange clustered services. Hmmm, that's a challenge. Or so one would think.

The Cheat
The client I was working for happened to have a 3rd party product (keeping read to find out) for Exchange that in essence allowed the "cheating". And I mean this in a very good way. It saved me a LOT of time. Meaning, we told the 3rd party product to take over all the existing Exchange services (MAPI, SMTP, OWA, IMAP, etc) and data for the Outlook, OWA, ActiveSync, & BlackBerry was available to all users. All their data was still available. This took a few minutes (3 or 4 minutes) on the switch-over. Now everyone was operating off the appliance and end users didn't know this besides restarting Outlook and re-authenticating to OWA (ActiveSync & BES users had a slight delay. BES users could be out of service for up to 15 minutes, but that's a limitation of BES). Once the appliance took over, we copied over the Exchange databases (.edb/.stm's) using robocopy to the new Exchange 2003 Server. We considered upgrading to 2007, but the appliance and all the associated Exchange applications would have had to been upgraded, and it wasn't cost effective (TCO reminder). So, after we started robocopy-ing, we went home.

Day 2 of the Migration & Failback
I'm not going to go into the details, but migrating took a few hours including getting the SSL certs for OWA and handling all that. Once the new hardware was setup with Exchange, it was time to bring back all the new email. As I previously said, the reason to copy over the databases to the new server, was the appliance then doesn't need to copy over all data, and just new email/data. Once we copied over the databases and transaction logs, we were able to get the Exchange Server fully operational and enable the failback from the appliance. We then failed back from the appliance to the new Exchange Server. This took a lot longer to check that all data was copied from the appliance to the new Exchange Server. This took 10 hours or so and then everyone had to relaunch Outlook and re-authenticate against the new server.

Appliance Details
The "cheat" was an Exchange high availability appliance from Teneros. For the record, this company has been bankrupt and while still in operation as of 7/10, I do NOT recommend purchasing this product. Even though this appliance runs 2 operating systems, Linux and Windows, the entire configuration is on 2 web pages. Meaning, the Teneros support team is really what runs this product, not the Exchange admin. As per the web interface, to say the amount of information and configuration is sparse, is putting it lightly. Overall the product worked well and we ran into 2 glitches due to permissions and resetting process of the AD name due to poor documentation. And the migration process took longer than expected since the status of synchronization is not very accurate. Not a big deal, since end users are working during the failover and failback. Overall solution is very impressive, but I have some doubts since I'm not a big fan of trusting secret functionality of a black box type solution. I like to know how applications work and I do have concerns over Exchange updates or patches breaking the Teneros functionality. If you are curious, pricing is around $10k, give or take a few thousands. If you wanted to see the demo, Teneros did present at the NY Exchange User Group meeting back in November of December of 2007 or check out their website.

-Ben

Microsoft (online/print) and My Photos

noreply@blogger.com (Ben Serebin) — Sun, 26 Dec 2010 05:21:00 +0000

Hello All,

Think you've seen me on the Microsoft website, you have! A couple of people have asked me about this. So, yes, it's true. Microsoft hired me about 9 months ago for a photo shoot. So, you'll see me online and in print media. Here's an example someone sent me recently.

I was hired for the look of an "IT manager". One of the coolest scenes was inside the train control center in Newark Penn Station. Anyone sees that, let me know and take a screen-shot.

Thanks,
-Ben

The Truth Behind ActiveSync & Enterprise Licensing

noreply@blogger.com (Ben Serebin) — Sun, 26 Dec 2010 04:57:00 +0000

Hello All,

Sadly I see this happen more often than it should. A potential client was lied to by another IT vendor's sales team. They claimed that for "iPhone ActiveSync remote wipe functionality, the Enterprise Client Access Licenses were required for Exchange Server 2010". Turns out this is a lie! Oops... caught by me.

The Enterprise Client Access License (aka eCAL) does not relate/control remote wipe for ActiveSync. The eCAL adds the following control for ActiveSync clients that support these features. Both server and client side need to support these ActiveSync policies for them to be in effect. Make sure you read that last sentence twice. I've put that in bold for you. This is critical in your understanding of ActiveSync device features. Pretty much each device that includes ActiveSync offers varying levels of ActiveSync capabilities.

iPhone support remote wipe (they also offer a limited capabilities for ActiveSync). The available ActiveSync functionality on the iPhone is documented here.

List of Exchange Server 2010 Enterprise Client Access License (eCAL) Functionality Abilities (screen-shots below)
Allow removable storage
Allow camera
Allow Wi-FI
Allow infrared
Allow Internet sharing from the device
Allow remote desktop from the device
Allow synchronization from a desktop
Allow Bluetooth
Allow browser
Allow consumer mail
Allow unsigned applications
Allow unsigned installation packages

-Ben